| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/09/24 05:44 | flow | patch-triage |
1mArgs:null Results: {
"EnableConfigs": [
"WPROBE_EVENTS",
"WPROBE_TRIGGERS",
"PROBE_EVENTS_BTF_ARGS"
],
"FocusSymbols": [
"trace_wprobe_create",
"wprobe_perf_handler",
"wprobe_trace_handler",
"wprobe_trigger",
"wprobe_trigger_cmd_parse",
"wprobe_work_func",
"modify_local_hw_breakpoint_addr",
"arch_modify_local_hw_breakpoint_addr"
],
"KMSANReasoning": "The patch series adds support for watchpoint probe (wprobe) dynamic tracing events and event triggers based on hardware breakpoints, along with x86 architecture support for modifying local hardware breakpoint addresses (using a per-CPU sequence counter `cpu_dr7_seq` to serialize `local_db_save`/`local_db_restore`).\n\nKMSAN vs. KASAN applicability analysis:\n1. Memory initialization: All heap structures (`trace_wprobe`, `wprobe_trigger_data`, `traceprobe_parse_context`) are allocated using zeroing allocators (`kzalloc_flex`, `kzalloc_obj`), and strings use `kstrdup`.\n2. Stack variables: In `arch/x86/include/asm/debugreg.h` and callers (`exc_machine_check_*`, `exc_nmi`, `exc_debug_kernel`), the newly introduced `dr7` and `dr7_seq` output pointers are written unconditionally on the first iteration of `local_db_save()` before any read or use in `local_db_restore()`. In `arch_modify_local_hw_breakpoint_addr()` and `kernel/trace/trace_wprobe.c`, all local variables are explicitly initialized before use.\n3. User-space data exposure / padding: No structures are sent to user space via `copy_to_user`, netlink, socket options, or ioctl outputs. Trace records written to the ftrace ring buffer use `struct wprobe_trace_entry_head`, which consists of `struct trace_entry` (8 bytes) and `unsigned long ip` (8 bytes), having no structure padding. Trace output is serialized as text through the standard tracing sequencer.\n4. Failure modes: Any potential bugs in this patch (such as concurrency issues across IRQ/NMI contexts, hardware breakpoint slot tracking, refcounting/lifetime of probe objects, or string parsing bounds) are memory corruption, concurrency, or control flow issues properly covered by KASAN, LOCKDEP, and standard kernel debugging tools.\n\nTherefore, the changes do not introduce or expose uninitialized memory risks, and a dedicated KMSAN session is not warranted.",
"NeedsKMSAN": false,
"Reasoning": "The patch introduces watchpoint probe (wprobe) dynamic event tracing and triggers (trace_wprobe.c), allowing userspace to configure hardware watchpoints on kernel memory addresses through the tracefs dynamic_events and trigger interfaces. It also adds new hardware breakpoint manipulation infrastructure (modify_local_hw_breakpoint_addr, arch_modify_local_hw_breakpoint_addr) and modifies x86 DR7 register synchronization across traps, NMIs, and MCEs. These changes introduce significant new parsing, concurrency, and hardware breakpoint handling logic directly exposed to userspace fuzzing.",
"WorthFuzzing": true
} |
| 1/1 | 2026/09/24 05:44 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit aa59edb7da5ec405cf1653e89aee6e32a2266101\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Thu Sep 24 05:44:54 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/Documentation/trace/index.rst b/Documentation/trace/index.rst\nindex f4058e8e92e3d..4f3fa60e4ecd9 100644\n--- a/Documentation/trace/index.rst\n+++ b/Documentation/trace/index.rst\n@@ -36,6 +36,7 @@ the Linux kernel.\n kprobes\n kprobetrace\n fprobetrace\n+ wprobetrace\n eprobetrace\n fprobe\n ring-buffer-design\ndiff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wprobetrace.rst\nnew file mode 100644\nindex 0000000000000..1d9b32d02d53b\n--- /dev/null\n+++ b/Documentation/trace/wprobetrace.rst\n@@ -0,0 +1,200 @@\n+.. SPDX-License-Identifier: GPL-2.0\n+\n+=======================================\n+Watchpoint probe (wprobe) Event Tracing\n+=======================================\n+\n+.. Author: Masami Hiramatsu \u003cmhiramat@kernel.org\u003e\n+\n+Overview\n+--------\n+\n+Wprobe event is a dynamic event based on the hardware breakpoint, which is\n+similar to other probe events, but it is for watching data access. It allows\n+you to trace which code accesses a specified data.\n+\n+As same as other dynamic events, wprobe events are defined via\n+`dynamic_events` interface file on tracefs.\n+\n+Synopsis of wprobe-events\n+-------------------------\n+::\n+\n+ w:[GRP/][EVENT] SPEC [FETCHARGS] : Probe on data access\n+\n+ GRP : Group name for wprobe. If omitted, use \"wprobes\" for it.\n+ EVENT : Event name for wprobe. If omitted, an event name is\n+ generated based on the address or symbol.\n+ SPEC : Breakpoint specification.\n+ [r|w|rw]@\u003cADDRESS|SYMBOL[+|-OFFS]\u003e[:LENGTH]\n+\n+ r|w|rw : Access type, r for read, w for write, and rw for both.\n+ Default is rw if omitted.\n+ ADDRESS : Address to watch (hexadecimal). MUST be in kernel space.\n+ SYMBOL[+|-OFFS] : Symbol name to watch. (Optional positive/negative offset)\n+ LENGTH : Length of the data to watch in bytes. (1, 2, 4, or 8)\n+ Default is 4.\n+\n+ FETCHARGS : Arguments. Each probe can have up to 128 args.\n+ $addr : Fetch the accessing address.\n+ $value : Fetch the memory value at the accessing address (same as +0($addr)).\n+ @ADDR : Fetch memory at ADDR (ADDR should be in kernel)\n+ @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol)\n+ +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\\*1)(\\*2)\n+ \\IMM : Store an immediate value to the argument.\n+ NAME=FETCHARG : Set NAME as the argument name of FETCHARG.\n+ FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types\n+ (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types\n+ (x8/x16/x32/x64), \"char\", \"string\", \"ustring\", \"symbol\", \"symstr\"\n+ and bitfield are supported.\n+ (STRUCT[,ASGN])FETCHARG-\u003eMEMBER[-\u003eMEMBER] : If BTF is supported, typecast\n+ FETCHARG to a pointer to STRUCT and then dereference the\n+ pointer defined by -\u003eMEMBER. ASGN can be specified optionally.\n+ If ASGN is specified, FETCHARG will be cast to the same offset\n+ position as the ASGN member, rather than to the beginning of\n+ the STRUCT.\n+ (STRUCT[,ASGN])(FETCHARG)-\u003eMEMBER[-\u003eMEMBER] : typecast can nest, so the above\n+ can also be used with another FETCHARG.\n+\n+ (\\*1) This is useful for fetching a field of data structures.\n+ (\\*2) \"u\" means user-space dereference.\n+\n+For the details of TYPE, see :ref:`kprobetrace documentation \u003ckprobetrace_types\u003e`.\n+\n+Usage examples\n+--------------\n+Here is an example to add a wprobe event on a variable `jiffies`.\n+::\n+\n+ # echo 'w:my_jiffies w@jiffies' \u003e\u003e dynamic_events\n+ # cat dynamic_events\n+ w:wprobes/my_jiffies w@jiffies\n+ # echo 1 \u003e events/wprobes/enable\n+ # cat trace | head\n+ # TASK-PID CPU# ||||| TIMESTAMP FUNCTION\n+ # | | | ||||| | |\n+ \u003cidle\u003e-0 [000] d.Z1. 717.026259: my_jiffies: (tick_do_update_jiffies64+0xbe/0x130)\n+ \u003cidle\u003e-0 [000] d.Z1. 717.026373: my_jiffies: (tick_do_update_jiffies64+0xbe/0x130)\n+\n+You can see the code which writes to `jiffies` is `tick_do_update_jiffies64()`.\n+\n+Notes\n+-----\n+Wprobe event does not disable itself even if the module is unloaded.\n+For example, if you add a wprobe event on a module variable, and then\n+unload the module, the wprobe event will still be enabled. This is for\n+watching the address is used unexpectedly after the module is unloaded.\n+\n+Combination with trigger action\n+-------------------------------\n+The event trigger action can extend the utilization of this wprobe.\n+\n+- set_wprobe:WPEVENT:FIELD[+|-ADJUST][:COUNT]\n+- set_wprobe:WPEVENT:(STRUCT[,ASGN])EVENT_FIELD-\u003eMEMBER[+|-ADJUST][:COUNT]\n+- clear_wprobe:WPEVENT[:FIELD[+|-ADJUST][:COUNT]]\n+- clear_wprobe:WPEVENT[:(STRUCT[,ASGN])EVENT_FIELD-\u003eMEMBER[+|-ADJUST][:COUNT]]\n+\n+Set these triggers to the target event, then the WPROBE event will be\n+setup to trace the memory access at FIELD[+|-ADJUST] address.\n+If BTF is supported, the target address can also be resolved using BTF\n+struct offset resolution: `(STRUCT[,ASGN])EVENT_FIELD-\u003eMEMBER[+|-ADJUST]`.\n+Here, EVENT_FIELD is an event field containing a pointer to STRUCT (or to\n+the ASGN member if ASGN is specified), and MEMBER is the struct member whose\n+offset is resolved automatically via BTF.\n+When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is\n+forcibly cleared. If FIELD[+|-ADJUST] is set, it clears WPEVENT only\n+if its watching address is the same as the FIELD[+|-ADJUST] value.\n+If COUNT is specified, it will set/clear WPEVENT only if it hits COUNT\n+times.\n+\n+Notes:\n+- set_wprobe only works on the wprobe which is NOT set a valid address yet,\n+ and it must be enabled after the trigger is set.\n+- clear_wprobe only works on the wprobe which is set a valid address, and it\n+ will be soft-disabled after the trigger is cleared.\n+- Therefore, if a trigger sets/clears a wprobe, other/same trigger events\n+ will not work (on the same event) while the wprobe is set.\n+\n+The set_wprobe trigger does not change the type and length, these\n+must be set when creating a new wprobe.\n+\n+The WPROBE event must be disabled when setting the new trigger\n+and it will be busy afterwards. Recommended usage is to add a new\n+wprobe at invalid dummy address (-1) and keep disabled.\n+\n+Wprobe triggers only support target addresses in kernel memory. If a\n+set_wprobe trigger evaluates to a user-space memory address or NULL\n+pointer, the trigger action ignores the update and skips setting the\n+watchpoint.\n+\n+Wprobe triggers are not supported on kprobe_events, because kprobes\n+themselves can use software breakpoints which conflicts with wprobe\n+operation.\n+\n+\n+For example, trace the first 8 bytes of the dentry data structure passed\n+to do_truncate() until it is deleted by dentry_kill().\n+(Note: all tracefs setup uses '\u003e\u003e' so that it does not kick do_truncate())\n+::\n+\n+ # echo 'w:watch rw@-1:8 address=$addr value=+0($addr)' \u003e\u003e dynamic_events\n+ # echo 'f:truncate do_truncate dentry=$arg2' \u003e\u003e dynamic_events\n+ # echo 'set_wprobe:watch:dentry' \u003e\u003e events/fprobes/truncate/trigger\n+ # echo 'f:dentry_kill dentry_kill dentry=$arg1' \u003e\u003e dynamic_events\n+ # echo 'clear_wprobe:watch:dentry' \u003e\u003e events/fprobes/dentry_kill/trigger\n+ # echo 1 \u003e\u003e events/fprobes/truncate/enable\n+ # echo 1 \u003e\u003e events/fprobes/dentry_kill/enable\n+\n+ # echo aaa \u003e /tmp/hoge\n+ # echo bbb \u003e /tmp/hoge\n+ # echo ccc \u003e /tmp/hoge\n+ # rm /tmp/hoge\n+\n+Then, the trace data will show::\n+\n+ # tracer: nop\n+ #\n+ # entries-in-buffer/entries-written: 32/32 #P:8\n+ #\n+ # _-----=\u003e irqs-off/BH-disabled\n+ # / _----=\u003e need-resched\n+ # | / _---=\u003e hardirq/softirq\n+ # || / _--=\u003e preempt-depth\n+ # ||| / _-=\u003e migrate-disable\n+ # |||| / delay\n+ # TASK-PID CPU# ||||| TIMESTAMP FUNCTION\n+ # | | | ||||| | |\n+ sh-107 [004] ...1. 9.990418: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618\n+ sh-107 [004] ...1. 9.990914: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b3de78\n+ sh-107 [004] ...1. 9.993175: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049ddd40\n+ sh-107 [004] ..... 9.995198: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8\n+ sh-107 [004] ...1. 9.995389: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db998\n+ sh-107 [004] ..Zff 9.997503: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080\n+ sh-107 [004] ..Zff 9.997509: watch: (path_openat+0x211/0xda0) address=0xffff8880048083a8 value=0x8200080\n+ sh-107 [004] ..Zff 9.997514: watch: (path_openat+0xa56/0xda0) address=0xffff8880048083a8 value=0x8200080\n+ sh-107 [004] ..Zff 9.997518: watch: (path_openat+0xae2/0xda0) address=0xffff8880048083a8 value=0x8200080\n+ sh-107 [004] ..... 9.997521: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8\n+ sh-107 [004] ...1. 9.997582: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004808270\n+ sh-107 [004] ...1. 9.999365: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db728\n+ sh-107 [004] ...1. 9.999388: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b1c000\n+ rm-113 [005] ..Zff 10.000965: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] ..Zff 10.000971: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] ..Zff 10.000984: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] ..Zff 10.000988: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] ..Zff 10.001010: watch: (lookup_one_qstr_excl+0x28/0x140) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] ..Zff 10.001014: watch: (lookup_one_qstr_excl+0xd1/0x140) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] ..Zff 10.001018: watch: (may_delete_dentry+0x1c/0x200) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] ..Zff 10.001021: watch: (may_delete_dentry+0x195/0x200) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] ..Zff 10.001031: watch: (vfs_unlink+0x5e/0x260) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] d.Z.. 10.001067: watch: (d_make_discardable+0x1b/0x40) address=0xffff8880048083a8 value=0x8200080\n+ rm-113 [005] d.Z.. 10.001071: watch: (d_make_discardable+0x29/0x40) address=0xffff8880048083a8 value=0x200080\n+ rm-113 [005] ...1. 10.001072: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8\n+ rm-113 [005] ...1. 10.001218: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8\n+ sh-107 [004] ...1. 10.001416: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db110\n+ sh-107 [004] ...1. 10.001444: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db248\n+ sh-107 [004] ...1. 10.001500: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618\n+ sh-107 [004] ...1. 10.002067: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78\n+ sh-107 [004] ...1. 10.904920: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78\n+ sh-107 [004] ...1. 10.905129: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618\n+\n+You can see the watch event is correctly configured on the dentry.\ndiff --git a/arch/Kconfig b/arch/Kconfig\nindex 45c6577723623..5292bd05d5d4d 100644\n--- a/arch/Kconfig\n+++ b/arch/Kconfig\n@@ -474,6 +474,24 @@ config HAVE_MIXED_BREAKPOINTS_REGS\n \t Select this option if your arch implements breakpoints under the\n \t latter fashion.\n \n+config HAVE_POST_BREAKPOINT_HOOK\n+\tbool\n+\tdepends on HAVE_HW_BREAKPOINT\n+\thelp\n+\t Depending on the arch implementation of hardware breakpoints,\n+\t some of them provide breakpoint hook after the target memory\n+\t is modified.\n+\t Select this option if your arch implements breakpoints overflow\n+\t handler hooks after the target memory is modified.\n+\n+config HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR\n+\tbool\n+\tdepends on HAVE_HW_BREAKPOINT\n+\thelp\n+\t Select this if the architecture can modify the address of an\n+\t installed hardware breakpoint on the local CPU without releasing\n+\t and reserving its hardware slot.\n+\n config HAVE_USER_RETURN_NOTIFIER\n \tbool\n \ndiff --git a/arch/x86/Kconfig b/arch/x86/Kconfig\nindex 15fd9ec5ecacb..e4eaf0b607d30 100644\n--- a/arch/x86/Kconfig\n+++ b/arch/x86/Kconfig\n@@ -249,6 +249,7 @@ config X86\n \tselect HAVE_FUNCTION_TRACER\n \tselect HAVE_GCC_PLUGINS\n \tselect HAVE_HW_BREAKPOINT\n+\tselect HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR\n \tselect HAVE_IOREMAP_PROT\n \tselect HAVE_IRQ_EXIT_ON_IRQ_STACK\tif X86_64\n \tselect HAVE_IRQ_TIME_ACCOUNTING\n@@ -288,6 +289,7 @@ config X86\n \tselect MMU_GATHER_RCU_TABLE_FREE\n \tselect MMU_GATHER_MERGE_VMAS\n \tselect HAVE_POSIX_CPU_TIMERS_TASK_WORK\n+\tselect HAVE_POST_BREAKPOINT_HOOK\n \tselect HAVE_REGS_AND_STACK_ACCESS_API\n \tselect HAVE_RELIABLE_STACKTRACE\t\tif UNWINDER_ORC || STACK_VALIDATION\n \tselect HAVE_FUNCTION_ARG_ACCESS_API\ndiff --git a/arch/x86/include/asm/debugreg.h b/arch/x86/include/asm/debugreg.h\nindex 854d82b88ff4e..515d2d313d0c8 100644\n--- a/arch/x86/include/asm/debugreg.h\n+++ b/arch/x86/include/asm/debugreg.h\n@@ -18,6 +18,7 @@\n #define DR7_FIXED_1\t0x00000400\n \n DECLARE_PER_CPU(unsigned long, cpu_dr7);\n+DECLARE_PER_CPU(unsigned int, cpu_dr7_seq);\n \n #ifndef CONFIG_PARAVIRT_XXL\n /*\n@@ -125,40 +126,69 @@ static __always_inline bool hw_breakpoint_active(void)\n \n extern void hw_breakpoint_restore(void);\n \n-static __always_inline unsigned long local_db_save(void)\n+static __always_inline void local_db_save(unsigned long *dr7,\n+\t\t\t\t\t unsigned int *dr7_seq)\n {\n-\tunsigned long dr7;\n+\tbool retrying = false;\n \n-\tif (cpu_feature_enabled(X86_FEATURE_HYPERVISOR) \u0026\u0026 !hw_breakpoint_active())\n-\t\treturn 0;\n+\tdo {\n+\t\t/* Inner loop: get DR7 with checking cpu_dr7_seq. */\n+\t\t*dr7_seq = this_cpu_read(cpu_dr7_seq);\n \n-\tget_debugreg(dr7, 7);\n+\t\tif (unlikely(retrying)) {\n+\t\t\t*dr7 = this_cpu_read(cpu_dr7);\n+\t\t} else {\n+\t\t\tget_debugreg(*dr7, 7);\n \n-\t/* Architecturally set bit */\n-\tdr7 \u0026= ~DR7_FIXED_1;\n-\tif (dr7)\n-\t\tset_debugreg(DR7_FIXED_1, 7);\n+\t\t\t/* Architecturally set bit */\n+\t\t\t*dr7 \u0026= ~DR7_FIXED_1;\n+\t\t}\n \n-\t/*\n-\t * Ensure the compiler doesn't lower the above statements into\n-\t * the critical section; disabling breakpoints late would not\n-\t * be good.\n-\t */\n-\tbarrier();\n+\t\tbarrier();\n+\t\tif (unlikely(*dr7_seq != this_cpu_read(cpu_dr7_seq))) {\n+\t\t\tretrying = true;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\t/* Outer loop: clear DR7 and retry if sequence number is updated. */\n+\t\tif (*dr7)\n+\t\t\tset_debugreg(DR7_FIXED_1, 7);\n \n-\treturn dr7;\n+\t\t/*\n+\t\t * Ensure the compiler doesn't lower the above statements into\n+\t\t * the critical section; disabling breakpoints late would not\n+\t\t * be good.\n+\t\t */\n+\t\tbarrier();\n+\t\tretrying = true;\n+\t} while (unlikely(*dr7_seq != this_cpu_read(cpu_dr7_seq)));\n }\n \n-static __always_inline void local_db_restore(unsigned long dr7)\n+static __always_inline void local_db_restore(unsigned long dr7,\n+\t\t\t\t\t unsigned int dr7_seq)\n {\n+\tunsigned long val;\n+\tunsigned int seq;\n+\n \t/*\n \t * Ensure the compiler doesn't raise this statement into\n \t * the critical section; enabling breakpoints early would\n \t * not be good.\n \t */\n \tbarrier();\n-\tif (dr7)\n-\t\tset_debugreg(dr7, 7);\n+\tdo {\n+\t\tseq = this_cpu_read(cpu_dr7_seq);\n+\t\tif (seq == dr7_seq) {\n+\t\t\tif (!dr7)\n+\t\t\t\treturn;\n+\t\t\tval = dr7;\n+\t\t} else {\n+\t\t\tval = this_cpu_read(cpu_dr7);\n+\t\t}\n+\n+\t\tset_debugreg(val | DR7_FIXED_1, 7);\n+\t\tbarrier();\n+\t} while (unlikely(seq != this_cpu_read(cpu_dr7_seq)));\n }\n \n #ifdef CONFIG_CPU_SUP_AMD\ndiff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw_breakpoint.h\nindex 0bc931cd0698b..f35ec491f6dda 100644\n--- a/arch/x86/include/asm/hw_breakpoint.h\n+++ b/arch/x86/include/asm/hw_breakpoint.h\n@@ -59,6 +59,8 @@ extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,\n \n \n int arch_install_hw_breakpoint(struct perf_event *bp);\n+int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,\n+\t\t\t\t\t unsigned long addr);\n void arch_uninstall_hw_breakpoint(struct perf_event *bp);\n void hw_breakpoint_pmu_read(struct perf_event *bp);\n void hw_breakpoint_pmu_unthrottle(struct perf_event *bp);\ndiff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c\nindex ab469605fc893..b3669d6f88d4f 100644\n--- a/arch/x86/kernel/cpu/mce/core.c\n+++ b/arch/x86/kernel/cpu/mce/core.c\n@@ -2108,6 +2108,10 @@ bool filter_mce(struct mce *m)\n static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)\n {\n \tirqentry_state_t irq_state;\n+\tunsigned long dr7;\n+\tunsigned int dr7_seq;\n+\n+\tlocal_db_save(\u0026dr7, \u0026dr7_seq);\n \n \tWARN_ON_ONCE(user_mode(regs));\n \n@@ -2116,20 +2120,27 @@ static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)\n \t * mce_check_crashing_cpu() for details.\n \t */\n \tif (mca_cfg.initialized \u0026\u0026 mce_check_crashing_cpu())\n-\t\treturn;\n+\t\tgoto out;\n \n \tirq_state = irqentry_nmi_enter(regs);\n \n \tdo_machine_check(regs);\n \n \tirqentry_nmi_exit(regs, irq_state);\n+out:\n+\tlocal_db_restore(dr7, dr7_seq);\n }\n \n static __always_inline void exc_machine_check_user(struct pt_regs *regs)\n {\n+\tunsigned long dr7;\n+\tunsigned int dr7_seq;\n+\n \tirqentry_enter_from_user_mode(regs);\n \n+\tlocal_db_save(\u0026dr7, \u0026dr7_seq);\n \tdo_machine_check(regs);\n+\tlocal_db_restore(dr7, dr7_seq);\n \n \tirqentry_exit_to_user_mode(regs);\n }\n@@ -2138,21 +2149,13 @@ static __always_inline void exc_machine_check_user(struct pt_regs *regs)\n /* MCE hit kernel mode */\n DEFINE_IDTENTRY_MCE(exc_machine_check)\n {\n-\tunsigned long dr7;\n-\n-\tdr7 = local_db_save();\n \texc_machine_check_kernel(regs);\n-\tlocal_db_restore(dr7);\n }\n \n /* The user mode variant. */\n DEFINE_IDTENTRY_MCE_USER(exc_machine_check)\n {\n-\tunsigned long dr7;\n-\n-\tdr7 = local_db_save();\n \texc_machine_check_user(regs);\n-\tlocal_db_restore(dr7);\n }\n \n #ifdef CONFIG_X86_FRED\n@@ -2169,28 +2172,20 @@ DEFINE_IDTENTRY_MCE_USER(exc_machine_check)\n */\n DEFINE_FREDENTRY_MCE(exc_machine_check)\n {\n-\tunsigned long dr7;\n-\n-\tdr7 = local_db_save();\n \tif (user_mode(regs))\n \t\texc_machine_check_user(regs);\n \telse\n \t\texc_machine_check_kernel(regs);\n-\tlocal_db_restore(dr7);\n }\n #endif\n #else\n /* 32bit unified entry point */\n DEFINE_IDTENTRY_RAW(exc_machine_check)\n {\n-\tunsigned long dr7;\n-\n-\tdr7 = local_db_save();\n \tif (user_mode(regs))\n \t\texc_machine_check_user(regs);\n \telse\n \t\texc_machine_check_kernel(regs);\n-\tlocal_db_restore(dr7);\n }\n #endif\n \ndiff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c\nindex f846c15f21ca1..b413adc2bbbf4 100644\n--- a/arch/x86/kernel/hw_breakpoint.c\n+++ b/arch/x86/kernel/hw_breakpoint.c\n@@ -40,6 +40,8 @@\n DEFINE_PER_CPU(unsigned long, cpu_dr7);\n EXPORT_PER_CPU_SYMBOL(cpu_dr7);\n \n+/* Sequence number of the per-CPU DR7 state. */\n+DEFINE_PER_CPU(unsigned int, cpu_dr7_seq);\n /* Per cpu debug address registers values */\n static DEFINE_PER_CPU(unsigned long, cpu_debugreg[HBP_NUM]);\n \n@@ -97,38 +99,95 @@ int decode_dr7(unsigned long dr7, int bpnum, unsigned *len, unsigned *type)\n int arch_install_hw_breakpoint(struct perf_event *bp)\n {\n \tstruct arch_hw_breakpoint *info = counter_arch_bp(bp);\n-\tunsigned long *dr7;\n+\tunsigned int seq;\n \tint i;\n \n \tlockdep_assert_irqs_disabled();\n \n-\tfor (i = 0; i \u003c HBP_NUM; i++) {\n-\t\tstruct perf_event **slot = this_cpu_ptr(\u0026bp_per_reg[i]);\n+\tif (perf_guest_in_guest())\n+\t\treturn -EBUSY;\n \n-\t\tif (!*slot) {\n-\t\t\t*slot = bp;\n+\tfor (i = 0; i \u003c HBP_NUM; i++) {\n+\t\tif (!this_cpu_cmpxchg(bp_per_reg[i], NULL, bp))\n \t\t\tbreak;\n-\t\t}\n \t}\n \n \tif (WARN_ONCE(i == HBP_NUM, \"Can't find any breakpoint slot\"))\n \t\treturn -EBUSY;\n \n-\tset_debugreg(info-\u003eaddress, i);\n-\t__this_cpu_write(cpu_debugreg[i], info-\u003eaddress);\n+\tdo {\n+\t\tseq = this_cpu_inc_return(cpu_dr7_seq);\n+\t\tthis_cpu_write(cpu_debugreg[i], info-\u003eaddress);\n+\t\tbarrier();\n+\t\tset_debugreg(info-\u003eaddress, i);\n+\t\tif (info-\u003emask)\n+\t\t\tamd_set_dr_addr_mask(info-\u003emask, i);\n+\t\tthis_cpu_or(cpu_dr7, encode_dr7(i, info-\u003elen, info-\u003etype));\n+\t\tbarrier();\n+\t\tset_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);\n+\t\tbarrier();\n+\t} while (seq != this_cpu_read(cpu_dr7_seq));\n \n-\tdr7 = this_cpu_ptr(\u0026cpu_dr7);\n-\t*dr7 |= encode_dr7(i, info-\u003elen, info-\u003etype);\n+\treturn 0;\n+}\n \n-\t/*\n-\t * Ensure we first write cpu_dr7 before we set the DR7 register.\n-\t * This ensures an NMI never see cpu_dr7 0 when DR7 is not.\n-\t */\n-\tbarrier();\n+/*\n+ * Modify the address of an installed perf counter breakpoint on local CPU.\n+ *\n+ * This updates the debug address register and per-CPU shadow state without\n+ * changing slot allocation or DR7. The core layer (modify_local_hw_breakpoint_addr())\n+ * manages updating bp-\u003eattr.bp_addr and counter_arch_bp(bp)-\u003eaddress.\n+ *\n+ * Atomic: called with IRQs disabled and handles variables and registers\n+ * local to this CPU.\n+ */\n+int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,\n+\t\t\t\t\t unsigned long addr)\n+{\n+\tstruct arch_hw_breakpoint *info = counter_arch_bp(bp);\n+\tstruct arch_hw_breakpoint hw;\n+\tstruct perf_event_attr attr = bp-\u003eattr;\n+\tunsigned int seq;\n+\tint i, ret;\n+\n+\tlockdep_assert_irqs_disabled();\n \n-\tset_debugreg(*dr7, 7);\n-\tif (info-\u003emask)\n-\t\tamd_set_dr_addr_mask(info-\u003emask, i);\n+\tif (perf_guest_in_guest())\n+\t\treturn -EBUSY;\n+\n+\tattr.bp_addr = addr;\n+\tret = hw_breakpoint_arch_parse(bp, \u0026attr, \u0026hw);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tfor (i = 0; i \u003c HBP_NUM; i++) {\n+\t\tif (this_cpu_read(bp_per_reg[i]) == bp)\n+\t\t\tbreak;\n+\t}\n+\n+\tif (WARN_ONCE(i == HBP_NUM, \"Can't find any breakpoint slot\"))\n+\t\treturn -ENOENT;\n+\n+\tdo {\n+\t\tseq = this_cpu_inc_return(cpu_dr7_seq);\n+\n+\t\t/* Disable breakpoint in shadow DR7 and hardware DR7 */\n+\t\tthis_cpu_and(cpu_dr7, ~__encode_dr7(i, info-\u003elen, info-\u003etype));\n+\t\tbarrier();\n+\t\tset_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);\n+\t\tbarrier();\n+\n+\t\tthis_cpu_write(cpu_debugreg[i], addr);\n+\t\tbarrier();\n+\t\tset_debugreg(addr, i);\n+\t\tbarrier();\n+\n+\t\t/* Re-enable breakpoint in shadow DR7 and hardware DR7 */\n+\t\tthis_cpu_or(cpu_dr7, encode_dr7(i, info-\u003elen, info-\u003etype));\n+\t\tbarrier();\n+\t\tset_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);\n+\t\tbarrier();\n+\t} while (seq != this_cpu_read(cpu_dr7_seq));\n \n \treturn 0;\n }\n@@ -146,36 +205,34 @@ void arch_uninstall_hw_breakpoint(struct perf_event *bp)\n {\n \tstruct arch_hw_breakpoint *info = counter_arch_bp(bp);\n \tunsigned long dr7;\n+\tunsigned int seq;\n \tint i;\n \n \tlockdep_assert_irqs_disabled();\n \n \tfor (i = 0; i \u003c HBP_NUM; i++) {\n-\t\tstruct perf_event **slot = this_cpu_ptr(\u0026bp_per_reg[i]);\n-\n-\t\tif (*slot == bp) {\n-\t\t\t*slot = NULL;\n+\t\tif (this_cpu_read(bp_per_reg[i]) == bp)\n \t\t\tbreak;\n-\t\t}\n \t}\n \n \tif (WARN_ONCE(i == HBP_NUM, \"Can't find any breakpoint slot\"))\n \t\treturn;\n \n-\tdr7 = this_cpu_read(cpu_dr7);\n-\tdr7 \u0026= ~__encode_dr7(i, info-\u003elen, info-\u003etype);\n-\n-\tset_debugreg(dr7, 7);\n-\tif (info-\u003emask)\n-\t\tamd_set_dr_addr_mask(0, i);\n-\n-\t/*\n-\t * Ensure the write to cpu_dr7 is after we've set the DR7 register.\n-\t * This ensures an NMI never see cpu_dr7 0 when DR7 is not.\n-\t */\n-\tbarrier();\n-\n-\tthis_cpu_write(cpu_dr7, dr7);\n+\tdo {\n+\t\tseq = this_cpu_inc_return(cpu_dr7_seq);\n+\t\tdr7 = this_cpu_read(cpu_dr7);\n+\t\tdr7 \u0026= ~__encode_dr7(i, info-\u003elen, info-\u003etype);\n+\t\tset_debugreg(dr7 | DR7_FIXED_1, 7);\n+\t\tif (info-\u003emask)\n+\t\t\tamd_set_dr_addr_mask(0, i);\n+\t\tbarrier();\n+\t\tthis_cpu_and(cpu_dr7,\n+\t\t\t ~__encode_dr7(i, info-\u003elen, info-\u003etype));\n+\t\tbarrier();\n+\t} while (seq != this_cpu_read(cpu_dr7_seq));\n+\n+\tWARN_ONCE(this_cpu_cmpxchg(bp_per_reg[i], bp, NULL) != bp,\n+\t\t \"Can't release breakpoint slot\");\n }\n \n static int arch_bp_generic_len(int x86_len)\n@@ -309,13 +366,14 @@ static inline bool within_cpu_entry(unsigned long addr, unsigned long end)\n \t\t\t\tsizeof(struct tlb_state)))\n \t\t\treturn true;\n \n-\t\t/*\n-\t\t * When in guest (X86_FEATURE_HYPERVISOR), local_db_save()\n-\t\t * will read per-cpu cpu_dr7 before clear dr7 register.\n-\t\t */\n+\t\t/* local_db_save() reads this state before clearing DR7. */\n \t\tif (within_area(addr, end, (unsigned long)\u0026per_cpu(cpu_dr7, cpu),\n \t\t\t\tsizeof(cpu_dr7)))\n \t\t\treturn true;\n+\t\tif (within_area(addr, end,\n+\t\t\t\t(unsigned long)\u0026per_cpu(cpu_dr7_seq, cpu),\n+\t\t\t\tsizeof(cpu_dr7_seq)))\n+\t\t\treturn true;\n \t}\n \n \treturn false;\n@@ -483,12 +541,18 @@ void flush_ptrace_hw_breakpoint(struct task_struct *tsk)\n \n void hw_breakpoint_restore(void)\n {\n-\tset_debugreg(__this_cpu_read(cpu_debugreg[0]), 0);\n-\tset_debugreg(__this_cpu_read(cpu_debugreg[1]), 1);\n-\tset_debugreg(__this_cpu_read(cpu_debugreg[2]), 2);\n-\tset_debugreg(__this_cpu_read(cpu_debugreg[3]), 3);\n-\tset_debugreg(DR6_RESERVED, 6);\n-\tset_debugreg(__this_cpu_read(cpu_dr7), 7);\n+\tunsigned int seq;\n+\n+\tdo {\n+\t\tseq = this_cpu_inc_return(cpu_dr7_seq);\n+\t\tset_debugreg(this_cpu_read(cpu_debugreg[0]), 0);\n+\t\tset_debugreg(this_cpu_read(cpu_debugreg[1]), 1);\n+\t\tset_debugreg(this_cpu_read(cpu_debugreg[2]), 2);\n+\t\tset_debugreg(this_cpu_read(cpu_debugreg[3]), 3);\n+\t\tset_debugreg(DR6_RESERVED, 6);\n+\t\tset_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);\n+\t\tbarrier();\n+\t} while (seq != this_cpu_read(cpu_dr7_seq));\n }\n EXPORT_SYMBOL_FOR_KVM(hw_breakpoint_restore);\n \ndiff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c\nindex 3c9f60d6ca5a7..c3806f7bd3e83 100644\n--- a/arch/x86/kernel/nmi.c\n+++ b/arch/x86/kernel/nmi.c\n@@ -531,11 +531,12 @@ enum nmi_states {\n };\n static DEFINE_PER_CPU(enum nmi_states, nmi_state);\n static DEFINE_PER_CPU(unsigned long, nmi_cr2);\n-static DEFINE_PER_CPU(unsigned long, nmi_dr7);\n \n DEFINE_IDTENTRY_RAW(exc_nmi)\n {\n \tirqentry_state_t irq_state;\n+\tunsigned long dr7;\n+\tunsigned int dr7_seq;\n \tstruct nmi_stats *nsp = this_cpu_ptr(\u0026nmi_stats);\n \n \t/*\n@@ -572,7 +573,7 @@ DEFINE_IDTENTRY_RAW(exc_nmi)\n \t */\n \tsev_es_ist_enter(regs);\n \n-\tthis_cpu_write(nmi_dr7, local_db_save());\n+\tlocal_db_save(\u0026dr7, \u0026dr7_seq);\n \n \tirq_state = irqentry_nmi_enter(regs);\n \n@@ -594,7 +595,7 @@ DEFINE_IDTENTRY_RAW(exc_nmi)\n \n \tirqentry_nmi_exit(regs, irq_state);\n \n-\tlocal_db_restore(this_cpu_read(nmi_dr7));\n+\tlocal_db_restore(dr7, dr7_seq);\n \n \tsev_es_ist_exit();\n \ndiff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c\nindex 30aa8369957e9..018abe736285d 100644\n--- a/arch/x86/kernel/traps.c\n+++ b/arch/x86/kernel/traps.c\n@@ -1231,8 +1231,12 @@ static noinstr void exc_debug_kernel(struct pt_regs *regs, unsigned long dr6)\n \t * it results in an endless recursion and stack overflow. Thus we stay\n \t * with the IDT approach, i.e., save DR7 and disable #DB.\n \t */\n-\tunsigned long dr7 = local_db_save();\n-\tirqentry_state_t irq_state = irqentry_nmi_enter(regs);\n+\tunsigned long dr7;\n+\tunsigned int dr7_seq;\n+\tirqentry_state_t irq_state;\n+\n+\tlocal_db_save(\u0026dr7, \u0026dr7_seq);\n+\tirq_state = irqentry_nmi_enter(regs);\n \tinstrumentation_begin();\n \n \t/*\n@@ -1289,7 +1293,7 @@ static noinstr void exc_debug_kernel(struct pt_regs *regs, unsigned long dr6)\n \tinstrumentation_end();\n \tirqentry_nmi_exit(regs, irq_state);\n \n-\tlocal_db_restore(dr7);\n+\tlocal_db_restore(dr7, dr7_seq);\n }\n \n static noinstr void exc_debug_user(struct pt_regs *regs, unsigned long dr6)\ndiff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h\nindex db199d653dd1a..bf65c7fffd996 100644\n--- a/include/linux/hw_breakpoint.h\n+++ b/include/linux/hw_breakpoint.h\n@@ -81,6 +81,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,\n \t\t\t perf_overflow_handler_t triggered,\n \t\t\t void *context);\n \n+int modify_local_hw_breakpoint_addr(struct perf_event *bp,\n+\t\t\t\t unsigned long addr);\n+\n extern int register_perf_hw_breakpoint(struct perf_event *bp);\n extern void unregister_hw_breakpoint(struct perf_event *bp);\n extern void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events);\n@@ -124,6 +127,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,\n \t\t\t perf_overflow_handler_t triggered,\n \t\t\t void *context)\t\t{ return NULL; }\n static inline int\n+modify_local_hw_breakpoint_addr(struct perf_event *bp,\n+\t\t\t\tunsigned long addr) { return -EOPNOTSUPP; }\n+static inline int\n register_perf_hw_breakpoint(struct perf_event *bp)\t{ return -ENOSYS; }\n static inline void unregister_hw_breakpoint(struct perf_event *bp)\t{ }\n static inline void\ndiff --git a/include/linux/perf_event.h b/include/linux/perf_event.h\nindex 5842552294c19..194e69270460e 100644\n--- a/include/linux/perf_event.h\n+++ b/include/linux/perf_event.h\n@@ -1677,8 +1677,8 @@ struct perf_guest_info_callbacks {\n \tunsigned int\t\t\t(*state)(void);\n \tunsigned long\t\t\t(*get_ip)(void);\n \tunsigned int\t\t\t(*handle_intel_pt_intr)(void);\n-\n \tvoid\t\t\t\t(*handle_mediated_pmi)(void);\n+\tbool\t\t\t\t(*in_guest)(void);\n };\n \n #ifdef CONFIG_GUEST_PERF_EVENTS\n@@ -1689,6 +1689,7 @@ DECLARE_STATIC_CALL(__perf_guest_state, *perf_guest_cbs-\u003estate);\n DECLARE_STATIC_CALL(__perf_guest_get_ip, *perf_guest_cbs-\u003eget_ip);\n DECLARE_STATIC_CALL(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs-\u003ehandle_intel_pt_intr);\n DECLARE_STATIC_CALL(__perf_guest_handle_mediated_pmi, *perf_guest_cbs-\u003ehandle_mediated_pmi);\n+DECLARE_STATIC_CALL(__perf_guest_in_guest, *perf_guest_cbs-\u003ein_guest);\n \n static inline unsigned int perf_guest_state(void)\n {\n@@ -1710,6 +1711,11 @@ static inline void perf_guest_handle_mediated_pmi(void)\n \tstatic_call(__perf_guest_handle_mediated_pmi)();\n }\n \n+static inline bool perf_guest_in_guest(void)\n+{\n+\treturn static_call(__perf_guest_in_guest)();\n+}\n+\n extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);\n extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);\n \n@@ -1718,6 +1724,7 @@ extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callback\n static inline unsigned int perf_guest_state(void)\t\t { return 0; }\n static inline unsigned long perf_guest_get_ip(void)\t\t { return 0; }\n static inline unsigned int perf_guest_handle_intel_pt_intr(void) { return 0; }\n+static inline bool perf_guest_in_guest(void)\t\t\t { return false; }\n \n #endif /* !CONFIG_GUEST_PERF_EVENTS */\n \ndiff --git a/include/linux/trace_events.h b/include/linux/trace_events.h\nindex 5cbd09c8be8d5..f81a5308c1167 100644\n--- a/include/linux/trace_events.h\n+++ b/include/linux/trace_events.h\n@@ -337,6 +337,7 @@ enum {\n \tTRACE_EVENT_FL_UPROBE_BIT,\n \tTRACE_EVENT_FL_EPROBE_BIT,\n \tTRACE_EVENT_FL_FPROBE_BIT,\n+\tTRACE_EVENT_FL_WPROBE_BIT,\n \tTRACE_EVENT_FL_CUSTOM_BIT,\n \tTRACE_EVENT_FL_TEST_STR_BIT,\n };\n@@ -367,6 +368,7 @@ enum {\n \tTRACE_EVENT_FL_UPROBE\t\t= (1 \u003c\u003c TRACE_EVENT_FL_UPROBE_BIT),\n \tTRACE_EVENT_FL_EPROBE\t\t= (1 \u003c\u003c TRACE_EVENT_FL_EPROBE_BIT),\n \tTRACE_EVENT_FL_FPROBE\t\t= (1 \u003c\u003c TRACE_EVENT_FL_FPROBE_BIT),\n+\tTRACE_EVENT_FL_WPROBE\t\t= (1 \u003c\u003c TRACE_EVENT_FL_WPROBE_BIT),\n \tTRACE_EVENT_FL_CUSTOM\t\t= (1 \u003c\u003c TRACE_EVENT_FL_CUSTOM_BIT),\n \tTRACE_EVENT_FL_TEST_STR\t\t= (1 \u003c\u003c TRACE_EVENT_FL_TEST_STR_BIT),\n };\n@@ -736,6 +738,7 @@ enum event_trigger_type {\n \tETT_EVENT_HIST\t\t= (1 \u003c\u003c 4),\n \tETT_HIST_ENABLE\t\t= (1 \u003c\u003c 5),\n \tETT_EVENT_EPROBE\t= (1 \u003c\u003c 6),\n+\tETT_EVENT_WPROBE\t= (1 \u003c\u003c 7),\n };\n \n extern int filter_match_preds(struct event_filter *filter, void *rec);\ndiff --git a/kernel/events/core.c b/kernel/events/core.c\nindex a6c8e38a31104..c5dead7c5f71d 100644\n--- a/kernel/events/core.c\n+++ b/kernel/events/core.c\n@@ -7749,6 +7749,7 @@ DEFINE_STATIC_CALL_RET0(__perf_guest_state, *perf_guest_cbs-\u003estate);\n DEFINE_STATIC_CALL_RET0(__perf_guest_get_ip, *perf_guest_cbs-\u003eget_ip);\n DEFINE_STATIC_CALL_RET0(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs-\u003ehandle_intel_pt_intr);\n DEFINE_STATIC_CALL_RET0(__perf_guest_handle_mediated_pmi, *perf_guest_cbs-\u003ehandle_mediated_pmi);\n+DEFINE_STATIC_CALL_RET0(__perf_guest_in_guest, *perf_guest_cbs-\u003ein_guest);\n \n void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)\n {\n@@ -7767,6 +7768,9 @@ void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)\n \tif (cbs-\u003ehandle_mediated_pmi)\n \t\tstatic_call_update(__perf_guest_handle_mediated_pmi,\n \t\t\t\t cbs-\u003ehandle_mediated_pmi);\n+\n+\tif (cbs-\u003ein_guest)\n+\t\tstatic_call_update(__perf_guest_in_guest, cbs-\u003ein_guest);\n }\n EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks);\n \n@@ -7780,6 +7784,7 @@ void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)\n \tstatic_call_update(__perf_guest_get_ip, (void *)\u0026__static_call_return0);\n \tstatic_call_update(__perf_guest_handle_intel_pt_intr, (void *)\u0026__static_call_return0);\n \tstatic_call_update(__perf_guest_handle_mediated_pmi, (void *)\u0026__static_call_return0);\n+\tstatic_call_update(__perf_guest_in_guest, (void *)\u0026__static_call_return0);\n \tsynchronize_rcu();\n }\n EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);\ndiff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c\nindex 789add0c185a0..69bfb17b87c41 100644\n--- a/kernel/events/hw_breakpoint.c\n+++ b/kernel/events/hw_breakpoint.c\n@@ -888,6 +888,54 @@ void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events)\n }\n EXPORT_SYMBOL_GPL(unregister_wide_hw_breakpoint);\n \n+/**\n+ * modify_local_hw_breakpoint_addr - update a local breakpoint address\n+ * @bp: the hwbp perf event for this CPU\n+ * @addr: the new address for @bp\n+ *\n+ * Update only the address of an installed breakpoint on the local CPU without\n+ * releasing and reserving its hardware slot. The caller must update other CPUs.\n+ * Return 0, or -EOPNOTSUPP if the architecture does not support this operation.\n+ *\n+ * Note: bp-\u003eattr.bp_addr and counter_arch_bp(bp)-\u003eaddress are updated before\n+ * modifying hardware registers so that interrupt/NMI handlers and CPU entry\n+ * paths observe the new address during the transition. If an NMI hits the old\n+ * breakpoint address before the hardware register update finishes, the event\n+ * handler will see bp-\u003eattr.bp_addr already pointing to @addr. If the\n+ * architecture update fails, both are rolled back to the previous address.\n+ * Callers must be prepared for this behavior.\n+ */\n+#ifdef CONFIG_HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR\n+int modify_local_hw_breakpoint_addr(struct perf_event *bp,\n+\t\t\t\t unsigned long addr)\n+{\n+\tunsigned long old_addr;\n+\tint ret;\n+\n+\tlockdep_assert_irqs_disabled();\n+\n+\told_addr = bp-\u003eattr.bp_addr;\n+\tWRITE_ONCE(counter_arch_bp(bp)-\u003eaddress, addr);\n+\tWRITE_ONCE(bp-\u003eattr.bp_addr, addr);\n+\n+\tret = arch_modify_local_hw_breakpoint_addr(bp, addr);\n+\tif (ret) {\n+\t\tWRITE_ONCE(counter_arch_bp(bp)-\u003eaddress, old_addr);\n+\t\tWRITE_ONCE(bp-\u003eattr.bp_addr, old_addr);\n+\t\treturn ret;\n+\t}\n+\n+\treturn 0;\n+}\n+#else\n+int modify_local_hw_breakpoint_addr(struct perf_event *bp,\n+\t\t\t\t unsigned long addr)\n+{\n+\treturn -EOPNOTSUPP;\n+}\n+#endif\n+EXPORT_SYMBOL_GPL(modify_local_hw_breakpoint_addr);\n+\n /**\n * hw_breakpoint_is_used - check if breakpoints are currently used\n *\ndiff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig\nindex 0ab5916575a97..5fd8ed63c516d 100644\n--- a/kernel/trace/Kconfig\n+++ b/kernel/trace/Kconfig\n@@ -862,6 +862,30 @@ config EPROBE_EVENTS\n \t convert the type of an event field. For example, turn an\n \t address into a string.\n \n+config WPROBE_EVENTS\n+\tbool \"Enable wprobe-based dynamic events\"\n+\tdepends on TRACING\n+\tdepends on HAVE_HW_BREAKPOINT\n+\tdepends on HAVE_POST_BREAKPOINT_HOOK\n+\tselect PROBE_EVENTS\n+\tselect DYNAMIC_EVENTS\n+\thelp\n+\t This allows the user to add watchpoint tracing events based on\n+\t hardware breakpoints on the fly via the ftrace interface.\n+\n+\t Those events can be inserted wherever hardware breakpoints can be\n+\t set, and record accessed memory address and values.\n+\n+config WPROBE_TRIGGERS\n+\tdepends on WPROBE_EVENTS\n+\tdepends on HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR\n+\tbool\n+\tdefault y\n+\thelp\n+\t This adds an event trigger which will set the wprobe on a specific\n+\t field of an event. This allows user to trace the memory access of\n+\t an address pointed by the event field.\n+\n config BPF_EVENTS\n \tdepends on BPF_SYSCALL\n \tdepends on (KPROBE_EVENTS || UPROBE_EVENTS) \u0026\u0026 PERF_EVENTS\ndiff --git a/kernel/trace/Makefile b/kernel/trace/Makefile\nindex f934ff586bd43..141c8323de20b 100644\n--- a/kernel/trace/Makefile\n+++ b/kernel/trace/Makefile\n@@ -126,6 +126,7 @@ obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o\n obj-$(CONFIG_FPROBE) += fprobe.o\n obj-$(CONFIG_RETHOOK) += rethook.o\n obj-$(CONFIG_FPROBE_EVENTS) += trace_fprobe.o\n+obj-$(CONFIG_WPROBE_EVENTS) += trace_wprobe.o\n \n obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o\n obj-$(CONFIG_RV) += rv/\ndiff --git a/kernel/trace/trace.c b/kernel/trace/trace.c\nindex a946e0183fd1e..61d7d8bfb1577 100644\n--- a/kernel/trace/trace.c\n+++ b/kernel/trace/trace.c\n@@ -4295,7 +4295,7 @@ static const char readme_msg[] =\n \t\"\\t\\t\\t Write into this file to define/undefine new trace events.\\n\"\n #endif\n #if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS) || \\\n- defined(CONFIG_FPROBE_EVENTS)\n+\tdefined(CONFIG_FPROBE_EVENTS) || defined(CONFIG_WPROBE_EVENTS)\n \t\"\\t accepts: event-definitions (one definition per line)\\n\"\n #if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS)\n \t\"\\t Format: p[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cplace\u003e [\u003cargs\u003e]\\n\"\n@@ -4305,6 +4305,9 @@ static const char readme_msg[] =\n \t\"\\t f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\\n\"\n \t\"\\t t[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003ctracepoint\u003e [\u003cargs\u003e]\\n\"\n #endif\n+#ifdef CONFIG_WPROBE_EVENTS\n+\t\"\\t w[:[\u003cgroup\u003e/][\u003cevent\u003e]] [r|w|rw]@\u003caddr\u003e[:\u003clen\u003e] [\u003cargs\u003e]\\n\"\n+#endif\n #ifdef CONFIG_HIST_TRIGGERS\n \t\"\\t s:[synthetic/]\u003cevent\u003e \u003cfield\u003e [\u003cfield\u003e]\\n\"\n #endif\n@@ -4359,6 +4362,10 @@ static const char readme_msg[] =\n \t\"\\t trigger: traceon, traceoff\\n\"\n \t\"\\t enable_event:\u003csystem\u003e:\u003cevent\u003e\\n\"\n \t\"\\t disable_event:\u003csystem\u003e:\u003cevent\u003e\\n\"\n+#ifdef CONFIG_WPROBE_TRIGGERS\n+\t\"\\t set_wprobe:\u003cwprobe-event\u003e:\u003cfield\u003e[+|-\u003coffset\u003e]\\n\"\n+\t\"\\t clear_wprobe:\u003cwprobe-event\u003e[:\u003cfield\u003e[+|-\u003coffset\u003e]]\\n\"\n+#endif\n #ifdef CONFIG_HIST_TRIGGERS\n \t\"\\t enable_hist:\u003csystem\u003e:\u003cevent\u003e\\n\"\n \t\"\\t disable_hist:\u003csystem\u003e:\u003cevent\u003e\\n\"\ndiff --git a/kernel/trace/trace.h b/kernel/trace/trace.h\nindex 74a7a50d1e78f..a61f840b18bdb 100644\n--- a/kernel/trace/trace.h\n+++ b/kernel/trace/trace.h\n@@ -179,6 +179,11 @@ struct fexit_trace_entry_head {\n \tunsigned long\t\tret_ip;\n };\n \n+struct wprobe_trace_entry_head {\n+\tstruct trace_entry\tent;\n+\tunsigned long\t\tip;\n+};\n+\n #define TRACE_BUF_SIZE\t\t1024\n \n struct trace_array;\n@@ -1978,6 +1983,7 @@ trigger_data_alloc(struct event_command *cmd_ops, char *cmd, char *param,\n \t\t void *private_data);\n extern void trigger_data_free(struct event_trigger_data *data);\n extern int event_trigger_init(struct event_trigger_data *data);\n+extern void event_trigger_free(struct event_trigger_data *data);\n extern int trace_event_trigger_enable_disable(struct trace_event_file *file,\n \t\t\t\t\t int trigger_enable);\n extern void update_cond_flag(struct trace_event_file *file);\ndiff --git a/kernel/trace/trace_events_trigger.c b/kernel/trace/trace_events_trigger.c\nindex 149300cc5e8ad..120e6ec634b76 100644\n--- a/kernel/trace/trace_events_trigger.c\n+++ b/kernel/trace/trace_events_trigger.c\n@@ -589,7 +589,7 @@ int event_trigger_init(struct event_trigger_data *data)\n * Usually used directly as the @free method in event trigger\n * implementations.\n */\n-static void\n+void\n event_trigger_free(struct event_trigger_data *data)\n {\n \tif (WARN_ON_ONCE(data-\u003eref \u003c= 0))\ndiff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c\nindex 804442b2f7d2b..ccf489635ad7f 100644\n--- a/kernel/trace/trace_probe.c\n+++ b/kernel/trace/trace_probe.c\n@@ -20,7 +20,7 @@\n #undef C\n #define C(a, b)\t\tb\n \n-static const char *trace_probe_err_text[] = { ERRORS };\n+const char *trace_probe_err_text[] = { ERRORS };\n \n static const char *reserved_field_names[] = {\n \t\"common_type\",\n@@ -890,6 +890,16 @@ static int query_btf_struct(const char *sname, struct traceprobe_parse_context *\n \t\tctx-\u003estruct_btf = NULL;\n \t}\n \n+\tif (ctx-\u003ebtf) {\n+\t\tid = btf_find_by_name_kind(ctx-\u003ebtf, sname, BTF_KIND_STRUCT);\n+\t\tif (id \u003e 0) {\n+\t\t\tbtf_get(ctx-\u003ebtf);\n+\t\t\tctx-\u003estruct_btf = ctx-\u003ebtf;\n+\t\t\tctx-\u003elast_struct = btf_type_by_id(ctx-\u003estruct_btf, id);\n+\t\t\treturn 0;\n+\t\t}\n+\t}\n+\n \tid = bpf_find_btf_id(sname, BTF_KIND_STRUCT, \u0026btf);\n \tif (id \u003c 0)\n \t\treturn id;\n@@ -965,7 +975,8 @@ static int handle_typecast(char *arg, struct traceprobe_parse_context *ctx)\n \n \tif (!(tparg_is_event_probe(ctx-\u003eflags) ||\n \t tparg_is_function_entry(ctx-\u003eflags) ||\n-\t tparg_is_function_return(ctx-\u003eflags))) {\n+\t tparg_is_function_return(ctx-\u003eflags) ||\n+\t tparg_is_wprobe(ctx-\u003eflags))) {\n \t\ttrace_probe_log_err(ctx-\u003eoffset, NOSUP_BTFARG);\n \t\treturn -EOPNOTSUPP;\n \t}\n@@ -1403,6 +1414,23 @@ static int parse_probe_vars(char *orig_arg, const struct fetch_type *t,\n \t\t\treturn 0;\n \t}\n \n+\t/* wprobe only support \"$addr\" and \"$value\" variable */\n+\tif (ctx-\u003eflags \u0026 TPARG_FL_WPROBE) {\n+\t\tif (!strcmp(arg, \"addr\")) {\n+\t\t\tcode-\u003eop = FETCH_OP_BADDR;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tif (!strcmp(arg, \"value\")) {\n+\t\t\tcode-\u003eop = FETCH_OP_BADDR;\n+\t\t\tcode++;\n+\t\t\tcode-\u003eop = FETCH_OP_DEREF;\n+\t\t\tcode-\u003eoffset = 0;\n+\t\t\t*pcode = code;\n+\t\t\treturn 0;\n+\t\t}\n+\t\tgoto inval;\n+\t}\n+\n \tif (strcmp(arg, \"comm\") == 0 || strcmp(arg, \"COMM\") == 0) {\n \t\tcode-\u003eop = FETCH_OP_COMM;\n \t\treturn 0;\n@@ -1462,8 +1490,8 @@ static int parse_probe_arg_register(char *arg, struct fetch_insn *code,\n {\n \tint ret;\n \n-\tif (ctx-\u003eflags \u0026 (TPARG_FL_TEVENT | TPARG_FL_FPROBE)) {\n-\t\t/* eprobe and fprobe do not handle registers */\n+\tif (ctx-\u003eflags \u0026 (TPARG_FL_TEVENT | TPARG_FL_FPROBE | TPARG_FL_WPROBE)) {\n+\t\t/* eprobe, fprobe and wprobe do not handle registers */\n \t\ttrace_probe_log_err(ctx-\u003eoffset, BAD_VAR);\n \t\treturn -EINVAL;\n \t}\ndiff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h\nindex d1fb3520700fb..c54c554b1949d 100644\n--- a/kernel/trace/trace_probe.h\n+++ b/kernel/trace/trace_probe.h\n@@ -90,6 +90,7 @@ typedef int (*print_type_func_t)(struct trace_seq *, void *, void *);\n \tFETCH_OP(STACK, param),\t\t/* Stack: .param = index */\t\\\n \tFETCH_OP(STACKP, none),\t\t/* Stack pointer */\t\t\\\n \tFETCH_OP(RETVAL, none),\t\t/* Return value */\t\t\\\n+\tFETCH_OP(BADDR, none),\t\t/* Break address */\t\t\\\n \tFETCH_OP(IMM, imm),\t\t/* Immediate: .immediate */\t\\\n \tFETCH_OP(COMM, none),\t\t/* Current comm */\t\t\\\n \tFETCH_OP(CURRENT, none),\t/* Current task_struct address */\\\n@@ -420,6 +421,7 @@ static inline int traceprobe_get_entry_data_size(struct trace_probe *tp)\n #define TPARG_FL_USER BIT(4)\n #define TPARG_FL_FPROBE BIT(5)\n #define TPARG_FL_TPOINT BIT(6)\n+#define TPARG_FL_WPROBE BIT(7)\n #define TPARG_FL_LOC_MASK\tGENMASK(4, 0)\n \n static inline bool tparg_is_function_entry(unsigned int flags)\n@@ -437,6 +439,11 @@ static inline bool tparg_is_event_probe(unsigned int flags)\n \treturn !!(flags \u0026 TPARG_FL_TEVENT);\n }\n \n+static inline bool tparg_is_wprobe(unsigned int flags)\n+{\n+\treturn !!(flags \u0026 TPARG_FL_WPROBE);\n+}\n+\n /* Each typecast consumes nested level. So the max number of typecast is 8. */\n #define TRACEPROBE_MAX_NESTED_LEVEL 8\n \n@@ -546,6 +553,10 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,\n \tC(ARG_TOO_LONG,\t\t\"Argument expression is too long\"),\t\t\\\n \tC(ARRAY_NO_CLOSE,\t\"Array is not closed\"),\t\t\\\n \tC(ARRAY_TOO_BIG,\t\"Array number is too big\"),\t\t\\\n+\tC(BAD_ACCESS_ADDR,\t\"Invalid access memory address\"),\t\t\\\n+\tC(BAD_ACCESS_FMT,\t\"Access memory address requires @\"),\t\t\\\n+\tC(BAD_ACCESS_LEN,\t\"This memory access length is not supported\"),\t\\\n+\tC(BAD_ACCESS_TYPE,\t\"Bad memory access type\"),\t\t\t\\\n \tC(BAD_ADDR_SUFFIX,\t\"Invalid probed address suffix\"),\t\t\\\n \tC(BAD_ARG_NAME,\t\t\"Argument name must follow the same rules as C identifiers\"),\t\\\n \tC(BAD_ARG_NUM,\t\t\"Invalid argument number\"),\t\t\\\n@@ -628,7 +639,14 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,\n \tC(TYPECAST_NOT_EVENT,\t\"Typecasts are only for eprobe fields\"),\t\\\n \tC(TYPECAST_REQ_FIELD,\t\"Typecast requires a field access\"),\t\\\n \tC(TYPECAST_SYM_OFFSET,\t\"@SYM+/-OFFSET with typecast needs parentheses\"),\t\\\n-\tC(USED_ARG_NAME,\t\"This argument name is already used\"),\n+\tC(USED_ARG_NAME,\t\"This argument name is already used\"),\t\t\\\n+\tC(WPROBE_NO_MAXACT,\t\"Watchpoint probe does not support maxactive\"),\t\\\n+\tC(WPROBE_NO_SIBLING,\t\"Watchpoint probe does not support sibling probes\"), \\\n+\tC(WPROBE_ON_KPROBE,\t\"Wprobe trigger is not supported on kprobe event\"), \\\n+\tC(WPROBE_NOT_FOUND,\t\"Target wprobe event is not found\"),\t\t\\\n+\tC(WPROBE_BUSY,\t\t\"Target wprobe event is already enabled\"),\t\\\n+\tC(WPROBE_NEED_FIELD,\t\"Wprobe trigger requires a target field\"),\t\\\n+\tC(WPROBE_BAD_FIELD,\t\"Target field must be pointer size\"),\n \n #undef C\n #define C(a, b)\t\tTP_ERR_##a\n@@ -652,6 +670,8 @@ void __trace_probe_log_err(int offset, int err);\n \n DEFINE_FREE(trace_probe_log_clear, const char *, if (_T) trace_probe_log_clear())\n \n+extern const char *trace_probe_err_text[];\n+\n #define trace_probe_log_err(offs, err)\t\\\n \t__trace_probe_log_err(offs, TP_ERR_##err)\n \ndiff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c\nnew file mode 100644\nindex 0000000000000..121310b72522f\n--- /dev/null\n+++ b/kernel/trace/trace_wprobe.c\n@@ -0,0 +1,1668 @@\n+// SPDX-License-Identifier: GPL-2.0\n+/*\n+ * Hardware-breakpoint-based tracing events\n+ *\n+ * Copyright (C) 2023, Masami Hiramatsu \u003cmhiramat@kernel.org\u003e\n+ */\n+#define pr_fmt(fmt)\t\"trace_wprobe: \" fmt\n+\n+#include \u003clinux/atomic.h\u003e\n+#include \u003clinux/compiler.h\u003e\n+#include \u003clinux/cpu.h\u003e\n+#include \u003clinux/cpuhotplug.h\u003e\n+#include \u003clinux/errno.h\u003e\n+#include \u003clinux/hw_breakpoint.h\u003e\n+#include \u003clinux/kallsyms.h\u003e\n+#include \u003clinux/list.h\u003e\n+#include \u003clinux/module.h\u003e\n+#include \u003clinux/mutex.h\u003e\n+#include \u003clinux/perf_event.h\u003e\n+#include \u003clinux/rculist.h\u003e\n+#include \u003clinux/security.h\u003e\n+#include \u003clinux/spinlock.h\u003e\n+#include \u003clinux/tracepoint.h\u003e\n+#include \u003clinux/uaccess.h\u003e\n+#include \u003clinux/workqueue.h\u003e\n+#include \u003clinux/irq_work.h\u003e\n+#include \u003clinux/preempt.h\u003e\n+\n+#include \u003casm/ptrace.h\u003e\n+\n+#include \"trace.h\"\n+#include \"trace_btf.h\"\n+#include \"trace_dynevent.h\"\n+#include \"trace_probe.h\"\n+#include \"trace_probe_kernel.h\"\n+#include \"trace_probe_tmpl.h\"\n+#include \"trace_output.h\"\n+\n+#define WPROBE_EVENT_SYSTEM \"wprobes\"\n+\n+static int trace_wprobe_create(const char *raw_command);\n+static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev);\n+static int trace_wprobe_release(struct dyn_event *ev);\n+static bool trace_wprobe_is_busy(struct dyn_event *ev);\n+static bool trace_wprobe_match(const char *system, const char *event,\n+\t\t\t int argc, const char **argv, struct dyn_event *ev);\n+\n+static struct dyn_event_operations trace_wprobe_ops = {\n+\t.create = trace_wprobe_create,\n+\t.show = trace_wprobe_show,\n+\t.is_busy = trace_wprobe_is_busy,\n+\t.free = trace_wprobe_release,\n+\t.match = trace_wprobe_match,\n+};\n+\n+struct trace_wprobe {\n+\tstruct dyn_event\tdevent;\n+\tstruct list_head\tactive_list;\n+\tstruct perf_event * __percpu *bp_event;\n+\tunsigned long\t\taddr;\n+\tint\t\t\toffset;\n+\tint\t\t\tlen;\n+\tint\t\t\ttype;\n+\tconst char\t\t*symbol;\n+\traw_spinlock_t\t\tlock;\n+\tstruct irq_work\t\tirq_work;\n+\tstruct work_struct\twork;\n+\tatomic_t\t\tmissed;\n+\t/*\n+\t * work_pending is set to 1 before irq_work_queue() and cleared to 0\n+\t * after wprobe_work_func() finishes on_each_cpu(). This prevents a\n+\t * new trigger from overwriting tw-\u003eaddr while the work is propagating\n+\t * the old address to per-CPU debug registers via IPI.\n+\t */\n+\tatomic_t\t\twork_pending;\n+\tstruct trace_probe\ttp;\n+};\n+\n+static bool is_trace_wprobe(struct dyn_event *ev)\n+{\n+\treturn ev-\u003eops == \u0026trace_wprobe_ops;\n+}\n+\n+static struct trace_wprobe *to_trace_wprobe(struct dyn_event *ev)\n+{\n+\treturn container_of(ev, struct trace_wprobe, devent);\n+}\n+\n+static DEFINE_MUTEX(wprobe_mutex);\n+static LIST_HEAD(active_wprobes);\n+\n+#define for_each_trace_wprobe(pos, dpos)\t\t\t\\\n+\tfor_each_dyn_event(dpos)\t\t\t\t\\\n+\t\tif (is_trace_wprobe(dpos) \u0026\u0026 (pos = to_trace_wprobe(dpos)))\n+\n+static bool trace_wprobe_is_busy(struct dyn_event *ev)\n+{\n+\tstruct trace_wprobe *tw = to_trace_wprobe(ev);\n+\n+\treturn trace_probe_is_enabled(\u0026tw-\u003etp);\n+}\n+\n+static bool trace_wprobe_match(const char *system, const char *event,\n+\t\t\t int argc, const char **argv, struct dyn_event *ev)\n+{\n+\tstruct trace_wprobe *tw = to_trace_wprobe(ev);\n+\n+\tif (event[0] != '\\0' \u0026\u0026 strcmp(trace_probe_name(\u0026tw-\u003etp), event))\n+\t\treturn false;\n+\n+\tif (system \u0026\u0026 strcmp(trace_probe_group_name(\u0026tw-\u003etp), system))\n+\t\treturn false;\n+\n+\treturn trace_probe_match_command_args(\u0026tw-\u003etp, argc, argv);\n+}\n+\n+/*\n+ * Note that we don't verify the fetch_insn code, since it does not come\n+ * from user space.\n+ */\n+static int\n+process_fetch_insn(struct fetch_insn *code, void *rec, void *edata,\n+\t\t void *dest, void *base)\n+{\n+\tvoid *baddr = rec;\n+\tunsigned long val;\n+\tint ret;\n+\n+retry:\n+\t/* 1st stage: get value from context */\n+\tswitch (code-\u003eop) {\n+\tcase FETCH_OP_BADDR:\n+\t\tval = (unsigned long)baddr;\n+\t\tbreak;\n+\tcase FETCH_NOP_SYMBOL:\t/* Ignore a place holder */\n+\t\tcode++;\n+\t\tgoto retry;\n+\tdefault:\n+\t\tret = process_common_fetch_insn(code, \u0026val);\n+\t\tif (ret \u003c 0)\n+\t\t\treturn ret;\n+\t}\n+\tcode++;\n+\n+\treturn process_fetch_insn_bottom(code, val, dest, base);\n+}\n+NOKPROBE_SYMBOL(process_fetch_insn)\n+\n+static void wprobe_trace_handler(struct trace_wprobe *tw,\n+\t\t\t\t unsigned long addr,\n+\t\t\t\t struct pt_regs *regs,\n+\t\t\t\t struct trace_event_file *trace_file)\n+{\n+\tstruct wprobe_trace_entry_head *entry;\n+\tstruct trace_event_call *call = trace_probe_event_call(\u0026tw-\u003etp);\n+\tstruct trace_event_buffer fbuffer;\n+\tint dsize;\n+\n+\tif (WARN_ON_ONCE(call != trace_file-\u003eevent_call))\n+\t\treturn;\n+\n+\tif (trace_trigger_soft_disabled(trace_file))\n+\t\treturn;\n+\n+\tif (READ_ONCE(tw-\u003eaddr) != addr)\n+\t\treturn;\n+\n+\tdsize = __get_data_size(\u0026tw-\u003etp, (void *)addr, NULL);\n+\n+\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file,\n+\t\t\t\t\t sizeof(*entry) + tw-\u003etp.size + dsize);\n+\tif (!entry)\n+\t\treturn;\n+\n+\tentry-\u003eip = instruction_pointer(regs);\n+\tstore_trace_args(\u0026entry[1], \u0026tw-\u003etp, (void *)addr, NULL, sizeof(*entry), dsize);\n+\n+\tfbuffer.regs = regs;\n+\ttrace_event_buffer_commit(\u0026fbuffer);\n+}\n+\n+static void wprobe_perf_handler(struct perf_event *bp,\n+\t\t\t struct perf_sample_data *data,\n+\t\t\t struct pt_regs *regs)\n+{\n+\tstruct trace_wprobe *tw = bp-\u003eoverflow_handler_context;\n+\tstruct event_file_link *link;\n+\tunsigned long addr = bp-\u003eattr.bp_addr;\n+\n+\ttrace_probe_for_each_link_rcu(link, \u0026tw-\u003etp)\n+\t\twprobe_trace_handler(tw, addr, regs, link-\u003efile);\n+}\n+\n+static int __register_trace_wprobe(struct trace_wprobe *tw)\n+{\n+\tstruct perf_event_attr attr;\n+\tint i, ret;\n+\n+\tif (tw-\u003ebp_event)\n+\t\treturn -EINVAL;\n+\n+\tfor (i = 0; i \u003c tw-\u003etp.nr_args; i++) {\n+\t\tret = traceprobe_update_arg(\u0026tw-\u003etp.args[i]);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n+\n+\thw_breakpoint_init(\u0026attr);\n+\tattr.bp_addr = tw-\u003eaddr;\n+\tattr.bp_len = tw-\u003elen;\n+\tattr.bp_type = tw-\u003etype;\n+\n+\ttw-\u003ebp_event = register_wide_hw_breakpoint(\u0026attr, wprobe_perf_handler, tw);\n+\tif (IS_ERR_PCPU(tw-\u003ebp_event)) {\n+\t\tint ret = PTR_ERR_PCPU(tw-\u003ebp_event);\n+\n+\t\ttw-\u003ebp_event = NULL;\n+\t\treturn ret;\n+\t}\n+\n+\tmutex_lock(\u0026wprobe_mutex);\n+\tlist_add(\u0026tw-\u003eactive_list, \u0026active_wprobes);\n+\tmutex_unlock(\u0026wprobe_mutex);\n+\n+\treturn 0;\n+}\n+\n+static void __unregister_trace_wprobe(struct trace_wprobe *tw)\n+{\n+\tif (tw-\u003ebp_event) {\n+\t\tmutex_lock(\u0026wprobe_mutex);\n+\t\tlist_del_init(\u0026tw-\u003eactive_list);\n+\t\tmutex_unlock(\u0026wprobe_mutex);\n+\n+\t\tirq_work_sync(\u0026tw-\u003eirq_work);\n+\t\tcancel_work_sync(\u0026tw-\u003ework);\n+\t\tunregister_wide_hw_breakpoint(tw-\u003ebp_event);\n+\t\ttw-\u003ebp_event = NULL;\n+\t}\n+}\n+\n+static int wprobe_cpu_online(unsigned int cpu)\n+{\n+\tstruct trace_wprobe *tw;\n+\n+\tmutex_lock(\u0026wprobe_mutex);\n+\tlist_for_each_entry(tw, \u0026active_wprobes, active_list) {\n+\t\tif (tw-\u003ebp_event \u0026\u0026 !per_cpu(*tw-\u003ebp_event, cpu)) {\n+\t\t\tstruct perf_event_attr attr;\n+\t\t\tstruct perf_event *bp;\n+\n+\t\t\thw_breakpoint_init(\u0026attr);\n+\t\t\tattr.bp_addr = READ_ONCE(tw-\u003eaddr);\n+\t\t\tattr.bp_len = tw-\u003elen;\n+\t\t\tattr.bp_type = tw-\u003etype;\n+\n+\t\t\tbp = perf_event_create_kernel_counter(\u0026attr, cpu, NULL,\n+\t\t\t\t\t\t\t wprobe_perf_handler, tw);\n+\t\t\tif (!IS_ERR(bp))\n+\t\t\t\tper_cpu(*tw-\u003ebp_event, cpu) = bp;\n+\t\t}\n+\t}\n+\tmutex_unlock(\u0026wprobe_mutex);\n+\n+\treturn 0;\n+}\n+\n+static int wprobe_cpu_offline(unsigned int cpu)\n+{\n+\tstruct trace_wprobe *tw;\n+\n+\tmutex_lock(\u0026wprobe_mutex);\n+\tlist_for_each_entry(tw, \u0026active_wprobes, active_list) {\n+\t\tif (tw-\u003ebp_event) {\n+\t\t\tstruct perf_event *bp = per_cpu(*tw-\u003ebp_event, cpu);\n+\n+\t\t\tif (bp) {\n+\t\t\t\tper_cpu(*tw-\u003ebp_event, cpu) = NULL;\n+\t\t\t\tunregister_hw_breakpoint(bp);\n+\t\t\t}\n+\t\t}\n+\t}\n+\tmutex_unlock(\u0026wprobe_mutex);\n+\n+\treturn 0;\n+}\n+\n+static int trace_wprobe_update_local(struct trace_wprobe *tw, unsigned long addr)\n+{\n+\tstruct perf_event * __percpu *pevent;\n+\tstruct perf_event *bp;\n+\n+\tpevent = tw-\u003ebp_event;\n+\tif (!pevent)\n+\t\treturn -EINVAL;\n+\n+\tbp = *this_cpu_ptr(pevent);\n+\tif (!bp)\n+\t\treturn -EINVAL;\n+\n+\tif (bp-\u003eattr.bp_addr == addr)\n+\t\treturn 0;\n+\n+\treturn modify_local_hw_breakpoint_addr(bp, addr);\n+}\n+\n+static void wprobe_smp_update_func(void *info)\n+{\n+\tstruct trace_wprobe *tw = info;\n+\tunsigned long addr = READ_ONCE(tw-\u003eaddr);\n+\n+\tif (trace_wprobe_update_local(tw, addr))\n+\t\tatomic_inc(\u0026tw-\u003emissed);\n+}\n+\n+static void wprobe_work_func(struct work_struct *work)\n+{\n+\tstruct trace_wprobe *tw = container_of(work, struct trace_wprobe, work);\n+\n+\tcpus_read_lock();\n+\ton_each_cpu(wprobe_smp_update_func, tw, true);\n+\tcpus_read_unlock();\n+\t/*\n+\t * Clear work_pending after all CPUs have updated their local debug\n+\t * registers. A new trigger may now update tw-\u003eaddr and queue a new\n+\t * irq_work.\n+\t */\n+\tatomic_set(\u0026tw-\u003ework_pending, 0);\n+}\n+\n+static void wprobe_irq_work_func(struct irq_work *irq_work)\n+{\n+\tstruct trace_wprobe *tw = container_of(irq_work, struct trace_wprobe, irq_work);\n+\n+\tschedule_work(\u0026tw-\u003ework);\n+}\n+\n+static void free_trace_wprobe(struct trace_wprobe *tw)\n+{\n+\tif (tw) {\n+\t\tif (tw-\u003ework.func) {\n+\t\t\tirq_work_sync(\u0026tw-\u003eirq_work);\n+\t\t\tcancel_work_sync(\u0026tw-\u003ework);\n+\t\t}\n+\t\ttrace_probe_cleanup(\u0026tw-\u003etp);\n+\t\tkfree(tw-\u003esymbol);\n+\t\tkfree(tw);\n+\t}\n+}\n+DEFINE_FREE(free_trace_wprobe, struct trace_wprobe *,\n+\tif (!IS_ERR_OR_NULL(_T))\n+\t\tfree_trace_wprobe(_T))\n+\n+\n+static struct trace_wprobe *alloc_trace_wprobe(const char *group,\n+\t\t\t\t\t const char *event,\n+\t\t\t\t\t const char *symbol,\n+\t\t\t\t\t int offset,\n+\t\t\t\t\t unsigned long addr,\n+\t\t\t\t\t int len, int type, int nargs)\n+{\n+\tstruct trace_wprobe *tw __free(free_trace_wprobe) = NULL;\n+\tint ret;\n+\n+\ttw = kzalloc_flex(*tw, tp.args, nargs);\n+\tif (!tw)\n+\t\treturn ERR_PTR(-ENOMEM);\n+\n+\traw_spin_lock_init(\u0026tw-\u003elock);\n+\tINIT_LIST_HEAD(\u0026tw-\u003eactive_list);\n+\tinit_irq_work(\u0026tw-\u003eirq_work, wprobe_irq_work_func);\n+\tINIT_WORK(\u0026tw-\u003ework, wprobe_work_func);\n+\tatomic_set(\u0026tw-\u003emissed, 0);\n+\tatomic_set(\u0026tw-\u003ework_pending, 0);\n+\n+\tif (symbol) {\n+\t\ttw-\u003esymbol = kstrdup(symbol, GFP_KERNEL);\n+\t\tif (!tw-\u003esymbol)\n+\t\t\treturn ERR_PTR(-ENOMEM);\n+\t}\n+\ttw-\u003eoffset = offset;\n+\ttw-\u003eaddr = addr;\n+\ttw-\u003elen = len;\n+\ttw-\u003etype = type;\n+\n+\tret = trace_probe_init(\u0026tw-\u003etp, event, group, false, nargs);\n+\tif (ret \u003c 0)\n+\t\treturn ERR_PTR(ret);\n+\n+\tdyn_event_init(\u0026tw-\u003edevent, \u0026trace_wprobe_ops);\n+\treturn_ptr(tw);\n+}\n+\n+static struct trace_wprobe *find_trace_wprobe(const char *event,\n+\t\t\t\t\t const char *group)\n+{\n+\tstruct dyn_event *pos;\n+\tstruct trace_wprobe *tw;\n+\n+\tfor_each_trace_wprobe(tw, pos)\n+\t\tif (strcmp(trace_probe_name(\u0026tw-\u003etp), event) == 0 \u0026\u0026\n+\t\t strcmp(trace_probe_group_name(\u0026tw-\u003etp), group) == 0)\n+\t\t\treturn tw;\n+\treturn NULL;\n+}\n+\n+static enum print_line_t\n+print_wprobe_event(struct trace_iterator *iter, int flags,\n+\t\t struct trace_event *event)\n+{\n+\tstruct wprobe_trace_entry_head *field;\n+\tstruct trace_seq *s = \u0026iter-\u003eseq;\n+\tstruct trace_probe *tp;\n+\n+\tfield = (struct wprobe_trace_entry_head *)iter-\u003eent;\n+\ttp = trace_probe_primary_from_call(\n+\t\tcontainer_of(event, struct trace_event_call, event));\n+\tif (WARN_ON_ONCE(!tp))\n+\t\tgoto out;\n+\n+\ttrace_seq_printf(s, \"%s: (\", trace_probe_name(tp));\n+\n+\tif (!seq_print_ip_sym_offset(s, field-\u003eip, flags))\n+\t\tgoto out;\n+\n+\ttrace_seq_putc(s, ')');\n+\n+\tif (trace_probe_print_args(s, tp-\u003eargs, tp-\u003enr_args,\n+\t\t\t (u8 *)\u0026field[1], field) \u003c 0)\n+\t\tgoto out;\n+\n+\ttrace_seq_putc(s, '\\n');\n+out:\n+\treturn trace_handle_return(s);\n+}\n+\n+static int wprobe_event_define_fields(struct trace_event_call *event_call)\n+{\n+\tint ret;\n+\tstruct wprobe_trace_entry_head field;\n+\tstruct trace_probe *tp;\n+\n+\ttp = trace_probe_primary_from_call(event_call);\n+\tif (WARN_ON_ONCE(!tp))\n+\t\treturn -ENOENT;\n+\n+\tDEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);\n+\n+\treturn traceprobe_define_arg_fields(event_call, sizeof(field), tp);\n+}\n+\n+static struct trace_event_functions wprobe_funcs = {\n+\t.trace\t= print_wprobe_event\n+};\n+\n+static struct trace_event_fields wprobe_fields_array[] = {\n+\t{ .type = TRACE_FUNCTION_TYPE,\n+\t .define_fields = wprobe_event_define_fields },\n+\t{}\n+};\n+\n+static int wprobe_register(struct trace_event_call *event,\n+\t\t\t enum trace_reg type, void *data);\n+\n+static inline void init_trace_event_call(struct trace_wprobe *tw)\n+{\n+\tstruct trace_event_call *call = trace_probe_event_call(\u0026tw-\u003etp);\n+\n+\tcall-\u003eevent.funcs = \u0026wprobe_funcs;\n+\tcall-\u003eclass-\u003efields_array = wprobe_fields_array;\n+\tcall-\u003eflags = TRACE_EVENT_FL_WPROBE;\n+\tcall-\u003eclass-\u003ereg = wprobe_register;\n+}\n+\n+static int register_wprobe_event(struct trace_wprobe *tw)\n+{\n+\tinit_trace_event_call(tw);\n+\treturn trace_probe_register_event_call(\u0026tw-\u003etp);\n+}\n+\n+static int register_trace_wprobe_event(struct trace_wprobe *tw)\n+{\n+\tstruct trace_wprobe *old_tw;\n+\tint ret;\n+\n+\tguard(mutex)(\u0026event_mutex);\n+\n+\told_tw = find_trace_wprobe(trace_probe_name(\u0026tw-\u003etp),\n+\t\t\t\t trace_probe_group_name(\u0026tw-\u003etp));\n+\tif (old_tw) {\n+\t\t/*\n+\t\t * Wprobe does not support sibling probes because the event\n+\t\t * trigger (set_wprobe/clear_wprobe) identifies the target\n+\t\t * wprobe by its event name. Having multiple wprobes sharing\n+\t\t * the same event name would make the target ambiguous.\n+\t\t */\n+\t\ttrace_probe_log_set_index(0);\n+\t\ttrace_probe_log_err(0, WPROBE_NO_SIBLING);\n+\t\treturn -EBUSY;\n+\t}\n+\n+\tret = register_wprobe_event(tw);\n+\tif (ret) {\n+\t\ttrace_probe_log_set_index(0);\n+\t\tif (ret == -EEXIST)\n+\t\t\ttrace_probe_log_err(0, EVENT_EXIST);\n+\t\telse if (ret != -ENOMEM)\n+\t\t\ttrace_probe_log_err(0, FAIL_REG_PROBE);\n+\t\treturn ret;\n+\t}\n+\n+\tdyn_event_add(\u0026tw-\u003edevent, trace_probe_event_call(\u0026tw-\u003etp));\n+\treturn 0;\n+}\n+static int unregister_wprobe_event(struct trace_wprobe *tw)\n+{\n+\treturn trace_probe_unregister_event_call(\u0026tw-\u003etp);\n+}\n+\n+static int unregister_trace_wprobe(struct trace_wprobe *tw)\n+{\n+\tif (trace_probe_has_sibling(\u0026tw-\u003etp))\n+\t\tgoto unreg;\n+\n+\tif (trace_probe_is_enabled(\u0026tw-\u003etp))\n+\t\treturn -EBUSY;\n+\n+\tif (trace_event_dyn_busy(trace_probe_event_call(\u0026tw-\u003etp)))\n+\t\treturn -EBUSY;\n+\n+\tif (unregister_wprobe_event(tw))\n+\t\treturn -EBUSY;\n+\n+unreg:\n+\t__unregister_trace_wprobe(tw);\n+\tdyn_event_remove(\u0026tw-\u003edevent);\n+\ttrace_probe_unlink(\u0026tw-\u003etp);\n+\n+\treturn 0;\n+}\n+\n+static int enable_trace_wprobe(struct trace_event_call *call,\n+\t\t\t struct trace_event_file *file)\n+{\n+\tstruct trace_probe *tp;\n+\tstruct trace_wprobe *tw;\n+\tbool enabled;\n+\tint ret = 0;\n+\n+\ttp = trace_probe_primary_from_call(call);\n+\tif (WARN_ON_ONCE(!tp))\n+\t\treturn -ENODEV;\n+\tenabled = trace_probe_is_enabled(tp);\n+\n+\tif (file) {\n+\t\tret = trace_probe_add_file(tp, file);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t} else {\n+\t\ttrace_probe_set_flag(tp, TP_FLAG_PROFILE);\n+\t}\n+\n+\tif (!enabled) {\n+\t\tlist_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) {\n+\t\t\tret = __register_trace_wprobe(tw);\n+\t\t\tif (ret \u003c 0) {\n+\t\t\t\tstruct trace_wprobe *tmp;\n+\n+\t\t\t\tlist_for_each_entry(tmp, trace_probe_probe_list(tp), tp.list) {\n+\t\t\t\t\tif (tmp == tw)\n+\t\t\t\t\t\tbreak;\n+\t\t\t\t\t__unregister_trace_wprobe(tmp);\n+\t\t\t\t}\n+\t\t\t\tif (file)\n+\t\t\t\t\ttrace_probe_remove_file(tp, file);\n+\t\t\t\telse\n+\t\t\t\t\ttrace_probe_clear_flag(tp, TP_FLAG_PROFILE);\n+\t\t\t\treturn ret;\n+\t\t\t}\n+\t\t}\n+\t}\n+\n+\treturn 0;\n+}\n+\n+static int disable_trace_wprobe(struct trace_event_call *call,\n+\t\t\t\tstruct trace_event_file *file)\n+{\n+\tstruct trace_wprobe *tw;\n+\tstruct trace_probe *tp;\n+\n+\ttp = trace_probe_primary_from_call(call);\n+\tif (WARN_ON_ONCE(!tp))\n+\t\treturn -ENODEV;\n+\n+\tif (file) {\n+\t\tif (!trace_probe_get_file_link(tp, file))\n+\t\t\treturn -ENOENT;\n+\t\tif (!trace_probe_has_single_file(tp))\n+\t\t\tgoto out;\n+\t\ttrace_probe_clear_flag(tp, TP_FLAG_TRACE);\n+\t} else {\n+\t\ttrace_probe_clear_flag(tp, TP_FLAG_PROFILE);\n+\t}\n+\n+\tif (!trace_probe_is_enabled(tp)) {\n+\t\tlist_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) {\n+\t\t\t__unregister_trace_wprobe(tw);\n+\t\t}\n+\t}\n+\n+out:\n+\tif (file)\n+\t\ttrace_probe_remove_file(tp, file);\n+\n+\treturn 0;\n+}\n+\n+static int wprobe_register(struct trace_event_call *event,\n+\t\t\t enum trace_reg type, void *data)\n+{\n+\tstruct trace_event_file *file = data;\n+\n+\tswitch (type) {\n+\tcase TRACE_REG_REGISTER:\n+\t\treturn enable_trace_wprobe(event, file);\n+\tcase TRACE_REG_UNREGISTER:\n+\t\treturn disable_trace_wprobe(event, file);\n+\n+#ifdef CONFIG_PERF_EVENTS\n+\tcase TRACE_REG_PERF_REGISTER:\n+\tcase TRACE_REG_PERF_UNREGISTER:\n+\tcase TRACE_REG_PERF_OPEN:\n+\tcase TRACE_REG_PERF_CLOSE:\n+\tcase TRACE_REG_PERF_ADD:\n+\tcase TRACE_REG_PERF_DEL:\n+\t\treturn -EOPNOTSUPP;\n+#endif\n+\t}\n+\treturn 0;\n+}\n+\n+static bool trace_wprobe_is_valid_addr(unsigned long addr, int len)\n+{\n+\tif (addr \u003c TASK_SIZE)\n+\t\treturn false;\n+\n+\tif (addr \u0026 (len - 1))\n+\t\treturn false;\n+\n+\treturn true;\n+}\n+\n+#ifdef CONFIG_WPROBE_TRIGGERS\n+static u64 wprobe_trigger_clear_target __aligned(8);\n+#define WPROBE_DEFAULT_CLEAR_ADDRESS ((unsigned long)\u0026wprobe_trigger_clear_target)\n+#endif\n+\n+static int parse_address_spec(const char *spec, unsigned long *addr, int *type,\n+\t\t\t int *len, char **symbol, int *offset)\n+{\n+\tchar *_spec __free(kfree) = NULL;\n+\tint _len = HW_BREAKPOINT_LEN_4;\n+\tint _type = HW_BREAKPOINT_RW;\n+\tunsigned long _addr = 0;\n+\tint _offset = 0;\n+\tchar *at, *col;\n+\n+\t_spec = kstrdup(spec, GFP_KERNEL);\n+\tif (!_spec)\n+\t\treturn -ENOMEM;\n+\n+\tat = strchr(_spec, '@');\n+\tcol = strchr(_spec, ':');\n+\n+\tif (!at) {\n+\t\ttrace_probe_log_err(0, BAD_ACCESS_FMT);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (at != _spec) {\n+\t\t*at = '\\0';\n+\n+\t\tif (strcmp(_spec, \"r\") == 0)\n+\t\t\t_type = HW_BREAKPOINT_R;\n+\t\telse if (strcmp(_spec, \"w\") == 0)\n+\t\t\t_type = HW_BREAKPOINT_W;\n+\t\telse if (strcmp(_spec, \"rw\") == 0)\n+\t\t\t_type = HW_BREAKPOINT_RW;\n+\t\telse {\n+\t\t\ttrace_probe_log_err(0, BAD_ACCESS_TYPE);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t}\n+\n+\tif (col) {\n+\t\t*col = '\\0';\n+\t\tif (kstrtoint(col + 1, 0, \u0026_len)) {\n+\t\t\ttrace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\n+\t\tswitch (_len) {\n+\t\tcase 1:\n+\t\t\t_len = HW_BREAKPOINT_LEN_1;\n+\t\t\tbreak;\n+\t\tcase 2:\n+\t\t\t_len = HW_BREAKPOINT_LEN_2;\n+\t\t\tbreak;\n+\t\tcase 4:\n+\t\t\t_len = HW_BREAKPOINT_LEN_4;\n+\t\t\tbreak;\n+\t\tcase 8:\n+\t\t\t_len = HW_BREAKPOINT_LEN_8;\n+\t\t\tbreak;\n+\t\tdefault:\n+\t\t\ttrace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t}\n+\n+#ifdef CONFIG_WPROBE_TRIGGERS\n+\tif (strcmp(at + 1, \"-1\") == 0) {\n+\t\t_addr = WPROBE_DEFAULT_CLEAR_ADDRESS;\n+\t} else if (kstrtoul(at + 1, 0, \u0026_addr) != 0) {\n+#else\n+\tif (kstrtoul(at + 1, 0, \u0026_addr) != 0) {\n+#endif\n+\t\tchar *off_str = strpbrk(at + 1, \"+-\");\n+\n+\t\tif (off_str) {\n+\t\t\tif (kstrtoint(off_str, 0, \u0026_offset) != 0) {\n+\t\t\t\ttrace_probe_log_err(off_str - _spec, BAD_PROBE_ADDR);\n+\t\t\t\treturn -EINVAL;\n+\t\t\t}\n+\t\t\t*off_str = '\\0';\n+\t\t}\n+\t\t_addr = kallsyms_lookup_name(at + 1);\n+\t\tif (!_addr) {\n+\t\t\ttrace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR);\n+\t\t\treturn -ENOENT;\n+\t\t}\n+\t\t_addr += _offset;\n+\t\t*symbol = kstrdup(at + 1, GFP_KERNEL);\n+\t\tif (!*symbol)\n+\t\t\treturn -ENOMEM;\n+\t}\n+\n+\tif (!trace_wprobe_is_valid_addr(_addr, _len)) {\n+\t\ttrace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\t*addr = _addr;\n+\t*offset = _offset;\n+\t*type = _type;\n+\t*len = _len;\n+\treturn 0;\n+}\n+\n+static int __trace_wprobe_create(int argc, const char *argv[])\n+{\n+\t/*\n+\t * Argument syntax:\n+\t * w[:[GRP/][EVENT]] SPEC\n+\t *\n+\t * SPEC:\n+\t * [r|w|rw]@[ADDR|SYMBOL[+OFFS]][:LEN]\n+\t */\n+\tstruct traceprobe_parse_context *ctx __free(traceprobe_parse_context) = NULL;\n+\tstruct trace_wprobe *tw __free(free_trace_wprobe) = NULL;\n+\tconst char *event = NULL, *group = WPROBE_EVENT_SYSTEM;\n+\tconst char *tplog __free(trace_probe_log_clear) = NULL;\n+\tchar *symbol __free(kfree) = NULL;\n+\tchar *gbuf __free(kfree) = NULL;\n+\tchar *ebuf __free(kfree) = NULL;\n+\tunsigned long addr;\n+\tint len, type, offset, i;\n+\tint ret;\n+\n+\tif (argv[0][0] != 'w')\n+\t\treturn -ECANCELED;\n+\n+\ttplog = trace_probe_log_init(\"wprobe\", argc, argv);\n+\n+\tif (argc \u003c 2) {\n+\t\ttrace_probe_log_set_index(0);\n+\t\ttrace_probe_log_err(0, NO_ARG_BODY);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tif (argv[0][1] != '\\0') {\n+\t\tif (argv[0][1] != ':') {\n+\t\t\ttrace_probe_log_set_index(0);\n+\t\t\ttrace_probe_log_err(1, WPROBE_NO_MAXACT);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t\tevent = \u0026argv[0][2];\n+\t}\n+\n+\ttrace_probe_log_set_index(1);\n+\tret = parse_address_spec(argv[1], \u0026addr, \u0026type, \u0026len, \u0026symbol, \u0026offset);\n+\tif (ret \u003c 0)\n+\t\treturn ret;\n+\n+\ttrace_probe_log_set_index(0);\n+\tif (event) {\n+\t\tgbuf = kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL);\n+\t\tif (!gbuf)\n+\t\t\treturn -ENOMEM;\n+\t\tret = traceprobe_parse_event_name(\u0026event, \u0026group, gbuf,\n+\t\t\t\t\t\t event - argv[0]);\n+\t\tif (ret)\n+\t\t\treturn ret;\n+\t}\n+\n+\tif (!event) {\n+\t\t/* Make a new event name */\n+\t\tebuf = kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL);\n+\t\tif (!ebuf)\n+\t\t\treturn -ENOMEM;\n+\t\tif (symbol)\n+\t\t\tsnprintf(ebuf, MAX_EVENT_NAME_LEN, \"%s\", symbol);\n+\t\telse\n+\t\t\tsnprintf(ebuf, MAX_EVENT_NAME_LEN, \"w_0x%lx\", addr);\n+\t\tsanitize_event_name(ebuf);\n+\t\tevent = ebuf;\n+\t}\n+\n+\targc -= 2; argv += 2;\n+\tif (argc \u003e MAX_TRACE_ARGS) {\n+\t\ttrace_probe_log_set_index(2);\n+\t\ttrace_probe_log_err(0, TOO_MANY_ARGS);\n+\t\treturn -E2BIG;\n+\t}\n+\ttw = alloc_trace_wprobe(group, event, symbol, offset, addr, len, type, argc);\n+\tif (IS_ERR(tw))\n+\t\treturn PTR_ERR(tw);\n+\n+\tctx = kzalloc_obj(*ctx);\n+\tif (!ctx)\n+\t\treturn -ENOMEM;\n+\n+\tctx-\u003eflags = TPARG_FL_KERNEL | TPARG_FL_WPROBE;\n+\n+\t/* parse arguments */\n+\tfor (i = 0; i \u003c argc; i++) {\n+\t\ttrace_probe_log_set_index(i + 2);\n+\t\tctx-\u003eoffset = 0;\n+\t\tret = traceprobe_parse_probe_arg(\u0026tw-\u003etp, i, argv[i], ctx);\n+\t\tif (ret)\n+\t\t\treturn ret;\t/* This can be -ENOMEM */\n+\t}\n+\n+\tret = traceprobe_set_print_fmt(\u0026tw-\u003etp, PROBE_PRINT_NORMAL);\n+\tif (ret \u003c 0)\n+\t\treturn ret;\n+\n+\tret = register_trace_wprobe_event(tw);\n+\tif (!ret)\n+\t\ttw = NULL; /* To avoid free */\n+\n+\treturn ret;\n+}\n+\n+static int trace_wprobe_create(const char *raw_command)\n+{\n+\treturn trace_probe_create(raw_command, __trace_wprobe_create);\n+}\n+\n+static int trace_wprobe_release(struct dyn_event *ev)\n+{\n+\tstruct trace_wprobe *tw = to_trace_wprobe(ev);\n+\tint ret = unregister_trace_wprobe(tw);\n+\n+\tif (!ret)\n+\t\tfree_trace_wprobe(tw);\n+\treturn ret;\n+}\n+\n+static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev)\n+{\n+\tstruct trace_wprobe *tw = to_trace_wprobe(ev);\n+\tconst char *type_str;\n+\tint i, len;\n+\n+\tseq_printf(m, \"w:%s/%s\", trace_probe_group_name(\u0026tw-\u003etp),\n+\t\t trace_probe_name(\u0026tw-\u003etp));\n+\n+\tif (tw-\u003etype == HW_BREAKPOINT_R)\n+\t\ttype_str = \"r\";\n+\telse if (tw-\u003etype == HW_BREAKPOINT_W)\n+\t\ttype_str = \"w\";\n+\telse\n+\t\ttype_str = \"rw\";\n+\n+\tif (tw-\u003elen == HW_BREAKPOINT_LEN_1)\n+\t\tlen = 1;\n+\telse if (tw-\u003elen == HW_BREAKPOINT_LEN_2)\n+\t\tlen = 2;\n+\telse if (tw-\u003elen == HW_BREAKPOINT_LEN_4)\n+\t\tlen = 4;\n+\telse\n+\t\tlen = 8;\n+\n+\tif (tw-\u003esymbol) {\n+\t\tif (tw-\u003eoffset)\n+\t\t\tseq_printf(m, \" %s@%s%+d:%d\", type_str, tw-\u003esymbol,\n+\t\t\t\t tw-\u003eoffset, len);\n+\t\telse\n+\t\t\tseq_printf(m, \" %s@%s:%d\", type_str, tw-\u003esymbol, len);\n+#ifdef CONFIG_WPROBE_TRIGGERS\n+\t} else if (tw-\u003eaddr == WPROBE_DEFAULT_CLEAR_ADDRESS) {\n+\t\tseq_printf(m, \" %s@-1:%d\", type_str, len);\n+#endif\n+\t} else {\n+\t\tseq_printf(m, \" %s@0x%lx:%d\", type_str, tw-\u003eaddr, len);\n+\t}\n+\n+\tfor (i = 0; i \u003c tw-\u003etp.nr_args; i++)\n+\t\tseq_printf(m, \" %s=%s\", tw-\u003etp.args[i].name, tw-\u003etp.args[i].comm);\n+\tseq_putc(m, '\\n');\n+\n+\treturn 0;\n+}\n+\n+static enum cpuhp_state wprobe_hp_state __ro_after_init;\n+\n+static __init int init_wprobe_trace(void)\n+{\n+\tint ret;\n+\n+\tret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, \"trace/wprobe:online\",\n+\t\t\t\twprobe_cpu_online, wprobe_cpu_offline);\n+\tif (ret \u003c 0)\n+\t\treturn ret;\n+\n+\twprobe_hp_state = ret;\n+\n+\tret = dyn_event_register(\u0026trace_wprobe_ops);\n+\tif (ret)\n+\t\tcpuhp_remove_state(wprobe_hp_state);\n+\n+\treturn ret;\n+}\n+fs_initcall(init_wprobe_trace);\n+\n+#ifdef CONFIG_WPROBE_TRIGGERS\n+\n+#define SET_WPROBE_STR\t\t\"set_wprobe\"\n+#define CLEAR_WPROBE_STR\t\"clear_wprobe\"\n+#define wprobe_trigger_log_err(file, glob, offs, err) \\\n+\ttracing_log_err((file)-\u003etr, \"wprobe_trigger\", glob, \\\n+\t\t\ttrace_probe_err_text, TP_ERR_##err, offs)\n+\n+struct wprobe_trigger_data {\n+\tstruct rcu_head\t\trcu;\n+\tstruct trace_event_file\t*file;\n+\tstruct trace_wprobe\t*tw;\n+\tint\t\t\toffset;\n+\tlong\t\t\tadjust;\n+\tconst char\t\t*field;\n+\tbool\t\t\tclear;\n+};\n+\n+static bool __wprobe_count_func(struct event_trigger_data *data,\n+\t\t\t\tunsigned long addr)\n+{\n+\tstruct wprobe_trigger_data *wprobe_data = data-\u003eprivate_data;\n+\tstruct trace_wprobe *tw = wprobe_data-\u003etw;\n+\tunsigned long cur_addr;\n+\n+\tif (in_nmi())\n+\t\treturn false;\n+\n+\tif (atomic_read(\u0026tw-\u003ework_pending))\n+\t\treturn false;\n+\n+\tcur_addr = READ_ONCE(tw-\u003eaddr);\n+\n+\tif (!wprobe_data-\u003eclear) {\n+\t\tif (cur_addr != WPROBE_DEFAULT_CLEAR_ADDRESS)\n+\t\t\treturn false;\n+\t\tif (!trace_wprobe_is_valid_addr(addr, tw-\u003elen))\n+\t\t\treturn false;\n+\t} else {\n+\t\tif (cur_addr == WPROBE_DEFAULT_CLEAR_ADDRESS)\n+\t\t\treturn false;\n+\t\tif (wprobe_data-\u003efield \u0026\u0026 cur_addr != addr)\n+\t\t\treturn false;\n+\t}\n+\n+\treturn true;\n+}\n+\n+static bool wprobe_count_func(struct event_trigger_data *data,\n+\t\t\t struct trace_buffer *buffer, void *rec,\n+\t\t\t struct ring_buffer_event *event)\n+{\n+\tstruct wprobe_trigger_data *wprobe_data = data-\u003eprivate_data;\n+\tstruct trace_wprobe *tw = wprobe_data-\u003etw;\n+\tunsigned long addr = 0;\n+\n+\tif (!data-\u003ecount)\n+\t\treturn false;\n+\n+\tif (wprobe_data-\u003efield) {\n+\t\tif (!rec)\n+\t\t\treturn false;\n+\t\taddr = *(unsigned long *)((char *)rec + wprobe_data-\u003eoffset);\n+\t\taddr += wprobe_data-\u003eadjust;\n+\t}\n+\n+\tif (!__wprobe_count_func(data, addr)) {\n+\t\tatomic_inc(\u0026tw-\u003emissed);\n+\t\treturn false;\n+\t}\n+\n+\tif (data-\u003ecount != -1)\n+\t\t(data-\u003ecount)--;\n+\n+\treturn true;\n+}\n+\n+static void wprobe_trigger(struct event_trigger_data *data,\n+\t\t\t struct trace_buffer *buffer, void *rec,\n+\t\t\t struct ring_buffer_event *event)\n+{\n+\tstruct wprobe_trigger_data *wprobe_data = data-\u003eprivate_data;\n+\tstruct trace_wprobe *tw = wprobe_data-\u003etw;\n+\tunsigned long target_addr, addr = 0, flags;\n+\n+\tif (wprobe_data-\u003efield) {\n+\t\tif (!rec)\n+\t\t\treturn;\n+\t\taddr = *(unsigned long *)((char *)rec + wprobe_data-\u003eoffset);\n+\t\taddr += wprobe_data-\u003eadjust;\n+\t}\n+\n+\traw_spin_lock_irqsave(\u0026tw-\u003elock, flags);\n+\n+\tif (!(data-\u003eflags \u0026 EVENT_TRIGGER_FL_COUNT)) {\n+\t\tif (!__wprobe_count_func(data, addr)) {\n+\t\t\tatomic_inc(\u0026tw-\u003emissed);\n+\t\t\tgoto out;\n+\t\t}\n+\t}\n+\n+\ttarget_addr = wprobe_data-\u003eclear ? WPROBE_DEFAULT_CLEAR_ADDRESS : addr;\n+\n+\t/*\n+\t * Try updating the hardware breakpoint on the local CPU first.\n+\t * If this fails, skip updating tw-\u003eaddr and do not queue irq_work.\n+\t * Note that this wprobe will not be traced until tw_addr is updated.\n+\t * (e.g. NMI happens on this CPU.)\n+\t */\n+\tif (trace_wprobe_update_local(tw, target_addr)) {\n+\t\tatomic_inc(\u0026tw-\u003emissed);\n+\t\tgoto out;\n+\t}\n+\n+\tWRITE_ONCE(tw-\u003eaddr, target_addr);\n+\tif (!wprobe_data-\u003eclear)\n+\t\tclear_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, \u0026wprobe_data-\u003efile-\u003eflags);\n+\telse\n+\t\tset_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, \u0026wprobe_data-\u003efile-\u003eflags);\n+\n+\t/*\n+\t * For other CPUs, kick irq_work to asynchronously propagate\n+\t * the new address.\n+\t */\n+\tif (num_online_cpus() \u003e 1) {\n+\t\tatomic_set(\u0026tw-\u003ework_pending, 1);\n+\t\tirq_work_queue(\u0026tw-\u003eirq_work);\n+\t}\n+\n+out:\n+\traw_spin_unlock_irqrestore(\u0026tw-\u003elock, flags);\n+}\n+\n+static void free_wprobe_trigger_data(struct wprobe_trigger_data *wprobe_data)\n+{\n+\tif (wprobe_data) {\n+\t\tkfree(wprobe_data-\u003efield);\n+\t\tkfree(wprobe_data);\n+\t}\n+}\n+DEFINE_FREE(free_wprobe_trigger_data, struct wprobe_trigger_data *, free_wprobe_trigger_data(_T));\n+\n+static void free_private_wprobe_trigger_data(struct event_trigger_data *data)\n+{\n+\tfree_wprobe_trigger_data(data-\u003eprivate_data);\n+}\n+\n+static int wprobe_trigger_print(struct seq_file *m,\n+\t\t\t struct event_trigger_data *data)\n+{\n+\tstruct wprobe_trigger_data *wprobe_data = data-\u003eprivate_data;\n+\tint missed = atomic_read(\u0026wprobe_data-\u003etw-\u003emissed);\n+\n+\tif (wprobe_data-\u003eclear) {\n+\t\tseq_printf(m, \"%s:%s\", CLEAR_WPROBE_STR,\n+\t\t\t trace_event_name(wprobe_data-\u003efile-\u003eevent_call));\n+\t\tif (wprobe_data-\u003efield) {\n+\t\t\tseq_printf(m, \":%s%+ld\",\n+\t\t\t\t wprobe_data-\u003efield, wprobe_data-\u003eadjust);\n+\t\t}\n+\t} else {\n+\t\tseq_printf(m, \"%s:%s:%s%+ld\", SET_WPROBE_STR,\n+\t\t\t trace_event_name(wprobe_data-\u003efile-\u003eevent_call),\n+\t\t\t wprobe_data-\u003efield, wprobe_data-\u003eadjust);\n+\t}\n+\n+\tif (data-\u003ecount == -1)\n+\t\tseq_puts(m, \":unlimited\");\n+\telse\n+\t\tseq_printf(m, \":count=%ld\", data-\u003ecount);\n+\n+\tif (data-\u003efilter_str)\n+\t\tseq_printf(m, \" if %s\", data-\u003efilter_str);\n+\n+\tif (missed)\n+\t\tseq_printf(m, \" # missed: %d\", missed);\n+\n+\tseq_putc(m, '\\n');\n+\n+\treturn 0;\n+}\n+\n+static struct wprobe_trigger_data *\n+wprobe_trigger_alloc(struct trace_wprobe *tw, struct trace_event_file *file,\n+\t\t bool clear)\n+{\n+\tstruct wprobe_trigger_data *wprobe_data;\n+\n+\twprobe_data = kzalloc_obj(*wprobe_data);\n+\tif (!wprobe_data)\n+\t\treturn NULL;\n+\n+\twprobe_data-\u003etw = tw;\n+\twprobe_data-\u003eclear = clear;\n+\twprobe_data-\u003efile = file;\n+\n+\treturn wprobe_data;\n+}\n+\n+static void wprobe_trigger_free(struct event_trigger_data *data)\n+{\n+\tstruct wprobe_trigger_data *wprobe_data = data-\u003eprivate_data;\n+\n+\tif (WARN_ON_ONCE(data-\u003eref \u003c= 0))\n+\t\treturn;\n+\n+\tdata-\u003eref--;\n+\tif (!data-\u003eref) {\n+\t\t/*\n+\t\t * Ensure SOFT_DISABLED flag is restored before clearing soft\n+\t\t * mode so that if the user did not explicitly enable this\n+\t\t * event, __ftrace_event_enable_disable() unregisters it.\n+\t\t */\n+\t\tset_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, \u0026wprobe_data-\u003efile-\u003eflags);\n+\t\t/* Remove the SOFT_MODE flag */\n+\t\ttrace_event_enable_disable(wprobe_data-\u003efile, 0, 1);\n+\t\ttrace_event_put_ref(wprobe_data-\u003efile-\u003eevent_call);\n+\t\ttrigger_data_free(data);\n+\t}\n+}\n+\n+#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS\n+\n+static int get_offset_of_field(struct btf *btf, const struct btf_type *type, char *field_name)\n+{\n+\tconst struct btf_member *field;\n+\tconst struct btf_type *mtype;\n+\tint bitoffs = 0;\n+\tu32 anon_offs;\n+\tchar *next;\n+\n+\tdo {\n+\t\tnext = strchr(field_name, '.');\n+\t\tif (next)\n+\t\t\t*next++ = '\\0';\n+\n+\t\tfield = btf_find_struct_member(btf, type, field_name, \u0026anon_offs, \u0026mtype);\n+\t\tif (IS_ERR_OR_NULL(field))\n+\t\t\treturn -ENOENT;\n+\n+\t\tif (btf_type_kflag(mtype)) {\n+\t\t\t/* Reject bitfield member access */\n+\t\t\tif (BTF_MEMBER_BITFIELD_SIZE(field-\u003eoffset))\n+\t\t\t\treturn -EINVAL;\n+\t\t\tbitoffs += anon_offs + BTF_MEMBER_BIT_OFFSET(field-\u003eoffset);\n+\t\t} else {\n+\t\t\tbitoffs += anon_offs + field-\u003eoffset;\n+\t\t}\n+\n+\t\tfield_name = next;\n+\t\tif (next) {\n+\t\t\ttype = btf_type_skip_modifiers(btf, field-\u003etype, NULL);\n+\t\t\tif (!type)\n+\t\t\t\treturn -ENOENT;\n+\t\t}\n+\t} while (next);\n+\treturn bitoffs / BITS_PER_BYTE;\n+}\n+\n+/* btf_put(NULL) is acceptable. */\n+DEFINE_FREE(btf_put, struct btf *, btf_put(_T))\n+\n+/* parse typecast: (TYPE[,ASGN])EVENT_FIELD-\u003eFIELD[.SUBFIELD...][+-OFFS] and set adjust. */\n+static int wprobe_trigger_typecast_parse(char *field_str,\n+\t\t\t\t\t struct trace_event_file *file,\n+\t\t\t\t\t struct wprobe_trigger_data *wprobe_data,\n+\t\t\t\t\t const char *glob)\n+{\n+\tstruct btf *btf __free(btf_put) = NULL;\n+\tchar *buf __free(kfree) = NULL;\n+\tstruct ftrace_event_field *field;\n+\tconst struct btf_type *type;\n+\tchar *assign_field;\n+\tchar *event_field;\n+\tchar *type_field;\n+\tchar *type_name;\n+\tchar *offs;\n+\tlong val = 0;\n+\tint base_offset = field_str - glob;\n+\tint event_field_offset;\n+\tint id, adjust;\n+\n+\tbuf = kstrdup(field_str, GFP_KERNEL);\n+\tif (!buf)\n+\t\treturn -ENOMEM;\n+\n+\ttype_name = buf + 1;\n+\tevent_field = strchr(type_name, ')');\n+\tif (!event_field) {\n+\t\twprobe_trigger_log_err(file, glob,\n+\t\t\t\t base_offset + (type_name - buf),\n+\t\t\t\t DEREF_OPEN_BRACE);\n+\t\treturn -EINVAL;\n+\t}\n+\t*event_field++ = '\\0';\n+\n+\t/* Check the optional assign field. */\n+\tassign_field = strchr(type_name, ',');\n+\tif (assign_field)\n+\t\t*assign_field++ = '\\0';\n+\n+\t/* Get the type field name. */\n+\ttype_field = strstr(event_field, \"-\u003e\");\n+\tif (!type_field) {\n+\t\twprobe_trigger_log_err(file, glob,\n+\t\t\t\t base_offset + (event_field - buf),\n+\t\t\t\t TYPECAST_REQ_FIELD);\n+\t\treturn -EINVAL;\n+\t}\n+\t*type_field = '\\0';\n+\ttype_field += 2;\n+\n+\toffs = strpbrk(type_field, \"+-\");\n+\tif (offs) {\n+\t\tif (kstrtol(offs, 0, \u0026val) \u003c 0) {\n+\t\t\twprobe_trigger_log_err(file, glob,\n+\t\t\t\t\t base_offset + (offs - buf),\n+\t\t\t\t\t BAD_DEREF_OFFS);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t\t*offs = '\\0';\n+\t}\n+\n+\t/* find type from BTF */\n+\tid = bpf_find_btf_id(type_name, BTF_KIND_STRUCT, \u0026btf);\n+\tif (id \u003c 0) {\n+\t\twprobe_trigger_log_err(file, glob,\n+\t\t\t\t base_offset + (type_name - buf),\n+\t\t\t\t BAD_BTF_TID);\n+\t\treturn id;\n+\t}\n+\n+\ttype = btf_type_by_id(btf, id);\n+\tif (!type) {\n+\t\twprobe_trigger_log_err(file, glob,\n+\t\t\t\t base_offset + (type_name - buf),\n+\t\t\t\t BAD_BTF_TID);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tadjust = get_offset_of_field(btf, type, type_field);\n+\tif (adjust \u003c 0) {\n+\t\twprobe_trigger_log_err(file, glob,\n+\t\t\t\t base_offset + (type_field - buf),\n+\t\t\t\t NO_BTF_FIELD);\n+\t\treturn adjust;\n+\t}\n+\twprobe_data-\u003eadjust = adjust + val;\n+\n+\tif (assign_field) {\n+\t\t/* assign_field should be a struct field */\n+\t\tadjust = get_offset_of_field(btf, type, assign_field);\n+\t\tif (adjust \u003c 0) {\n+\t\t\twprobe_trigger_log_err(file, glob,\n+\t\t\t\t\t base_offset + (assign_field - buf),\n+\t\t\t\t\t NO_BTF_FIELD);\n+\t\t\treturn adjust;\n+\t\t}\n+\t\twprobe_data-\u003eadjust -= adjust;\n+\t}\n+\n+\tevent_field_offset = base_offset + (event_field - buf);\n+\tfield = trace_find_event_field(file-\u003eevent_call, event_field);\n+\tif (!field) {\n+\t\twprobe_trigger_log_err(file, glob, event_field_offset, NO_EVENT_FIELD);\n+\t\treturn -ENOENT;\n+\t}\n+\tif (field-\u003esize != sizeof(void *)) {\n+\t\twprobe_trigger_log_err(file, glob, event_field_offset, WPROBE_BAD_FIELD);\n+\t\treturn -ENOEXEC;\n+\t}\n+\twprobe_data-\u003eoffset = field-\u003eoffset;\n+\twprobe_data-\u003efield = kstrdup(event_field, GFP_KERNEL);\n+\tif (!wprobe_data-\u003efield)\n+\t\treturn -ENOMEM;\n+\n+\treturn 0;\n+}\n+#else\n+static int wprobe_trigger_typecast_parse(char *field_str,\n+\t\t\t\t\t struct trace_event_file *file,\n+\t\t\t\t\t struct wprobe_trigger_data *wprobe_data,\n+\t\t\t\t\t const char *glob)\n+{\n+\twprobe_trigger_log_err(file, glob, field_str - glob, NOSUP_BTFARG);\n+\treturn -EOPNOTSUPP;\n+}\n+#endif /* CONFIG_PROBE_EVENTS_BTF_ARGS */\n+\n+static int wprobe_trigger_field_parse(char *field_str, struct trace_event_file *file,\n+\t\t\t\t\tstruct wprobe_trigger_data *wprobe_data,\n+\t\t\t\t\tconst char *glob)\n+{\n+\tstruct ftrace_event_field *field;\n+\tchar *offs;\n+\n+\tif (field_str[0] == '(')\n+\t\treturn wprobe_trigger_typecast_parse(field_str, file, wprobe_data, glob);\n+\n+\toffs = strpbrk(field_str, \"+-\");\n+\tif (offs) {\n+\t\tlong val;\n+\n+\t\tif (kstrtol(offs, 0, \u0026val) \u003c 0) {\n+\t\t\twprobe_trigger_log_err(file, glob, offs - glob, BAD_DEREF_OFFS);\n+\t\t\treturn -EINVAL;\n+\t\t}\n+\t\twprobe_data-\u003eadjust = val;\n+\t\t*offs = '\\0';\n+\t}\n+\n+\tfield = trace_find_event_field(file-\u003eevent_call, field_str);\n+\tif (!field) {\n+\t\twprobe_trigger_log_err(file, glob, field_str - glob, NO_EVENT_FIELD);\n+\t\treturn -ENOENT;\n+\t}\n+\tif (field-\u003esize != sizeof(void *)) {\n+\t\twprobe_trigger_log_err(file, glob, field_str - glob, WPROBE_BAD_FIELD);\n+\t\treturn -ENOEXEC;\n+\t}\n+\twprobe_data-\u003eoffset = field-\u003eoffset;\n+\twprobe_data-\u003efield = kstrdup(field_str, GFP_KERNEL);\n+\tif (!wprobe_data-\u003efield)\n+\t\treturn -ENOMEM;\n+\n+\treturn 0;\n+}\n+\n+static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops,\n+\t\t\t\t struct trace_event_file *file,\n+\t\t\t\t char *glob, char *cmd,\n+\t\t\t\t char *param_and_filter)\n+{\n+\t/*\n+\t * set_wprobe:EVENT:FIELD[+OFFS]\n+\t * clear_wprobe:EVENT[:FIELD[+OFFS]]\n+\t */\n+\tstruct wprobe_trigger_data *wprobe_data = NULL;\n+\tstruct event_trigger_data *trigger_data = NULL;\n+\tstruct trace_event_file *wprobe_file;\n+\tchar *event_str, *comment;\n+\tstruct trace_array *tr = file-\u003etr;\n+\tbool remove, clear = false;\n+\tunsigned long orig_addr = 0;\n+\tstruct trace_wprobe *tw;\n+\tchar *param, *filter;\n+\tint ret;\n+\n+\tremove = event_trigger_check_remove(glob);\n+\n+\tif (!strcmp(cmd, CLEAR_WPROBE_STR))\n+\t\tclear = true;\n+\n+\tif (param_and_filter) {\n+\t\t/* Recover original trigger string to show the error log correctly. */\n+\t\tif (*(param_and_filter - 1) == '\\0')\n+\t\t\t*(param_and_filter - 1) = ':';\n+\t\tcomment = strchr(param_and_filter, '#');\n+\t\tif (comment)\n+\t\t\t*comment = '\\0';\n+\t}\n+\n+\tif (event_trigger_empty_param(param_and_filter)) {\n+\t\twprobe_trigger_log_err(file, glob, strlen(cmd) + 1, WPROBE_NOT_FOUND);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\tret = event_trigger_separate_filter(param_and_filter, \u0026param, \u0026filter, true);\n+\tif (ret)\n+\t\treturn ret;\n+\n+\tif (file-\u003eevent_call-\u003eflags \u0026 TRACE_EVENT_FL_KPROBE) {\n+\t\twprobe_trigger_log_err(file, glob, 0, WPROBE_ON_KPROBE);\n+\t\treturn -EOPNOTSUPP;\n+\t}\n+\n+\tevent_str = strsep(\u0026param, \":\");\n+\n+\t/* Find target wprobe */\n+\ttw = find_trace_wprobe(event_str, WPROBE_EVENT_SYSTEM);\n+\tif (!tw) {\n+\t\twprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND);\n+\t\treturn -ENOENT;\n+\t}\n+\t/* The target wprobe must not be used (unless clear) */\n+\tif (!remove \u0026\u0026 !clear \u0026\u0026 trace_probe_is_enabled(\u0026tw-\u003etp)) {\n+\t\twprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_BUSY);\n+\t\treturn -EBUSY;\n+\t}\n+\n+\twprobe_file = find_event_file(tr, WPROBE_EVENT_SYSTEM, event_str);\n+\tif (!wprobe_file) {\n+\t\twprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND);\n+\t\treturn -EINVAL;\n+\t}\n+\n+\twprobe_data = wprobe_trigger_alloc(tw, wprobe_file, clear);\n+\tif (!wprobe_data)\n+\t\treturn -ENOMEM;\n+\n+\t/* clear_wprobe does not need field, but can have optional field. */\n+\tif (!clear) {\n+\t\tchar *field_str = strsep(\u0026param, \":\");\n+\n+\t\tif (!field_str) {\n+\t\t\twprobe_trigger_log_err(file, glob, strlen(glob), WPROBE_NEED_FIELD);\n+\t\t\tret = -EINVAL;\n+\t\t\tgoto out_free;\n+\t\t}\n+\t\tret = wprobe_trigger_field_parse(field_str, file, wprobe_data, glob);\n+\t\tif (ret \u003c 0)\n+\t\t\tgoto out_free;\n+\t} else if (param \u0026\u0026 (isalpha(param[0]) || param[0] == '_' || param[0] == '(')) {\n+\t\tif (strncmp(param, \"count=\", 6) != 0 \u0026\u0026\n+\t\t (strcmp(param, \"unlimited\") != 0 ||\n+\t\t trace_find_event_field(file-\u003eevent_call, \"unlimited\"))) {\n+\t\t\tchar *field_str = strsep(\u0026param, \":\");\n+\n+\t\t\tret = wprobe_trigger_field_parse(field_str, file, wprobe_data, glob);\n+\t\t\tif (ret \u003c 0)\n+\t\t\t\tgoto out_free;\n+\t\t}\n+\t}\n+\n+\tif (param) {\n+\t\tif (!strcmp(param, \"unlimited\"))\n+\t\t\tparam = NULL;\n+\t\telse if (!strncmp(param, \"count=\", 6))\n+\t\t\tparam += 6;\n+\t}\n+\n+\ttrigger_data = trigger_data_alloc(cmd_ops, cmd, param, wprobe_data);\n+\tif (!trigger_data) {\n+\t\tret = -ENOMEM;\n+\t\tgoto out_free;\n+\t}\n+\n+\ttrigger_data-\u003eprivate_data_free = free_private_wprobe_trigger_data;\n+\n+\tif (remove) {\n+\t\tevent_trigger_unregister(cmd_ops, file, glob+1, trigger_data);\n+\t\ttrigger_data_free(trigger_data);\n+\t\treturn 0;\n+\t}\n+\n+\t/* Up the trigger_data count to make sure nothing frees it on failure */\n+\tevent_trigger_init(trigger_data);\n+\n+\tret = event_trigger_parse_num(param, trigger_data);\n+\tif (ret) {\n+\t\twprobe_trigger_log_err(file, glob, param - glob, BAD_IMM);\n+\t\tgoto out_free_trigger;\n+\t}\n+\n+\tret = event_trigger_set_filter(cmd_ops, file, filter, trigger_data);\n+\tif (ret \u003c 0)\n+\t\tgoto out_free_trigger;\n+\n+\t/* Soft-enable (register) wprobe event on WPROBE_DEFAULT_CLEAR_ADDRESS */\n+\tif (!trace_event_try_get_ref(wprobe_file-\u003eevent_call)) {\n+\t\tret = -ENODEV;\n+\t\tgoto out_free_trigger;\n+\t}\n+\n+\tif (!clear) {\n+\t\torig_addr = tw-\u003eaddr;\n+\t\tWRITE_ONCE(tw-\u003eaddr, WPROBE_DEFAULT_CLEAR_ADDRESS);\n+\t}\n+\n+\tret = trace_event_enable_disable(wprobe_file, 1, 1);\n+\tif (ret \u003c 0) {\n+\t\tif (!clear)\n+\t\t\tWRITE_ONCE(tw-\u003eaddr, orig_addr);\n+\t\tgoto out_put;\n+\t}\n+\n+\tret = event_trigger_register(cmd_ops, file, glob, trigger_data);\n+\tif (ret)\n+\t\tgoto out_disable;\n+\n+\t/* Balance the event_trigger_init() at registration start */\n+\tevent_trigger_free(trigger_data);\n+\treturn 0;\n+\n+out_disable:\n+\tif (!clear)\n+\t\tWRITE_ONCE(tw-\u003eaddr, orig_addr);\n+\ttrace_event_enable_disable(wprobe_file, 0, 1);\n+out_put:\n+\ttrace_event_put_ref(wprobe_file-\u003eevent_call);\n+out_free_trigger:\n+\tevent_trigger_reset_filter(cmd_ops, trigger_data);\n+\ttrigger_data_free(trigger_data);\n+\treturn ret;\n+\n+out_free:\n+\tfree_wprobe_trigger_data(wprobe_data);\n+\treturn ret;\n+}\n+\n+/* Return event_trigger_data if there is a trigger which points the same wprobe */\n+static struct event_trigger_data *\n+wprobe_trigger_find_same(struct event_trigger_data *test,\n+\t\t\t struct trace_event_file *file)\n+{\n+\tstruct wprobe_trigger_data *test_wprobe_data = test-\u003eprivate_data;\n+\tstruct wprobe_trigger_data *wprobe_data;\n+\tstruct event_trigger_data *iter;\n+\n+\tlist_for_each_entry(iter, \u0026file-\u003etriggers, list) {\n+\t\twprobe_data = iter-\u003eprivate_data;\n+\t\tif (!wprobe_data ||\n+\t\t iter-\u003ecmd_ops-\u003etrigger_type !=\n+\t\t test-\u003ecmd_ops-\u003etrigger_type)\n+\t\t\tcontinue;\n+\t\tif (wprobe_data-\u003etw == test_wprobe_data-\u003etw \u0026\u0026\n+\t\t wprobe_data-\u003eclear == test_wprobe_data-\u003eclear)\n+\t\t\treturn iter;\n+\t}\n+\treturn NULL;\n+}\n+\n+static int wprobe_register_trigger(char *glob,\n+\t\t\t\t struct event_trigger_data *data,\n+\t\t\t\t struct trace_event_file *file)\n+{\n+\tint ret = 0;\n+\n+\tlockdep_assert_held(\u0026event_mutex);\n+\n+\t/* The same wprobe is not accept on the same file (event) */\n+\tif (wprobe_trigger_find_same(data, file))\n+\t\treturn -EEXIST;\n+\n+\tif (data-\u003ecmd_ops-\u003einit) {\n+\t\tret = data-\u003ecmd_ops-\u003einit(data);\n+\t\tif (ret \u003c 0)\n+\t\t\treturn ret;\n+\t}\n+\n+\tlist_add_rcu(\u0026data-\u003elist, \u0026file-\u003etriggers);\n+\n+\tupdate_cond_flag(file);\n+\tret = trace_event_trigger_enable_disable(file, 1);\n+\tif (ret \u003c 0) {\n+\t\tlist_del_rcu(\u0026data-\u003elist);\n+\t\tupdate_cond_flag(file);\n+\t\ttracepoint_synchronize_unregister();\n+\t\tif (data-\u003ecmd_ops-\u003efree)\n+\t\t\tdata-\u003ecmd_ops-\u003efree(data);\n+\t}\n+\treturn ret;\n+}\n+\n+static void wprobe_unregister_trigger(char *glob,\n+\t\t\t\t struct event_trigger_data *test,\n+\t\t\t\t struct trace_event_file *file)\n+{\n+\tstruct event_trigger_data *data;\n+\n+\tlockdep_assert_held(\u0026event_mutex);\n+\n+\tdata = wprobe_trigger_find_same(test, file);\n+\tif (!data)\n+\t\treturn;\n+\n+\tlist_del_rcu(\u0026data-\u003elist);\n+\ttrace_event_trigger_enable_disable(file, 0);\n+\tupdate_cond_flag(file);\n+\ttracepoint_synchronize_unregister();\n+\tif (data-\u003ecmd_ops-\u003efree)\n+\t\tdata-\u003ecmd_ops-\u003efree(data);\n+}\n+\n+static struct event_command trigger_wprobe_set_cmd = {\n+\t.name\t\t\t= SET_WPROBE_STR,\n+\t.trigger_type\t\t= ETT_EVENT_WPROBE,\n+\t/* This triggers after when the event is recorded. */\n+\t.flags\t\t\t= EVENT_CMD_FL_NEEDS_REC,\n+\t.parse\t\t\t= wprobe_trigger_cmd_parse,\n+\t.reg\t\t\t= wprobe_register_trigger,\n+\t.unreg\t\t\t= wprobe_unregister_trigger,\n+\t.set_filter\t\t= set_trigger_filter,\n+\t.trigger\t\t= wprobe_trigger,\n+\t.count_func\t\t= wprobe_count_func,\n+\t.print\t\t\t= wprobe_trigger_print,\n+\t.init\t\t\t= event_trigger_init,\n+\t.free\t\t\t= wprobe_trigger_free,\n+};\n+\n+static struct event_command trigger_wprobe_clear_cmd = {\n+\t.name\t\t\t= CLEAR_WPROBE_STR,\n+\t.trigger_type\t\t= ETT_EVENT_WPROBE,\n+\t/* This triggers after when the event is recorded. */\n+\t.flags\t\t\t= EVENT_CMD_FL_NEEDS_REC,\n+\t.parse\t\t\t= wprobe_trigger_cmd_parse,\n+\t.reg\t\t\t= wprobe_register_trigger,\n+\t.unreg\t\t\t= wprobe_unregister_trigger,\n+\t.set_filter\t\t= set_trigger_filter,\n+\t.trigger\t\t= wprobe_trigger,\n+\t.count_func\t\t= wprobe_count_func,\n+\t.print\t\t\t= wprobe_trigger_print,\n+\t.init\t\t\t= event_trigger_init,\n+\t.free\t\t\t= wprobe_trigger_free,\n+};\n+\n+static __init int init_trigger_wprobe_cmds(void)\n+{\n+\tint ret;\n+\n+\tret = register_event_command(\u0026trigger_wprobe_set_cmd);\n+\tif (WARN_ON(ret \u003c 0))\n+\t\treturn ret;\n+\tret = register_event_command(\u0026trigger_wprobe_clear_cmd);\n+\tif (WARN_ON(ret \u003c 0))\n+\t\tunregister_event_command(\u0026trigger_wprobe_set_cmd);\n+\n+\treturn ret;\n+}\n+fs_initcall(init_trigger_wprobe_cmds);\n+#endif /* CONFIG_WPROBE_TRIGGERS */\ndiff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config\nindex 544de0db5f58f..f067874902ed7 100644\n--- a/tools/testing/selftests/ftrace/config\n+++ b/tools/testing/selftests/ftrace/config\n@@ -1,5 +1,6 @@\n CONFIG_BPF_SYSCALL=y\n CONFIG_DEBUG_INFO_BTF=y\n+CONFIG_DEBUG_INFO_BTF_MODULES=y\n CONFIG_DEBUG_INFO_DWARF4=y\n CONFIG_EPROBE_EVENTS=y\n CONFIG_FPROBE=y\n@@ -27,3 +28,5 @@ CONFIG_STACK_TRACER=y\n CONFIG_TRACER_SNAPSHOT=y\n CONFIG_UPROBES=y\n CONFIG_UPROBE_EVENTS=y\n+CONFIG_WPROBE_EVENTS=y\n+CONFIG_WPROBE_TRIGGERS=y\ndiff --git a/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc\nnew file mode 100644\nindex 0000000000000..3c622fb07e7f9\n--- /dev/null\n+++ b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc\n@@ -0,0 +1,66 @@\n+#!/bin/sh\n+# SPDX-License-Identifier: GPL-2.0\n+# description: Generic dynamic event - add/remove wprobe events\n+# requires: dynamic_events \"w[:[\u003cgroup\u003e/][\u003cevent\u003e]] [r|w|rw]@\u003caddr\u003e[:\u003clen\u003e]\":README\n+\n+echo 0 \u003e events/enable\n+echo \u003e dynamic_events\n+\n+# Use jiffies as a variable that is frequently written to.\n+TARGET=jiffies\n+\n+echo \"w:my_wprobe w@$TARGET\" \u003e\u003e dynamic_events\n+\n+if ! grep -q my_wprobe dynamic_events; then\n+ echo \"Failed to create wprobe event\"\n+ exit_fail\n+fi\n+\n+if [ ! -d events/wprobes/my_wprobe ]; then\n+ echo \"Failed to create wprobe event directory\"\n+ exit_fail\n+fi\n+\n+if ! echo 1 \u003e events/wprobes/my_wprobe/enable; then\n+ echo \"Failed to enable wprobe (hardware watchpoints may be exhausted)\"\n+ exit_unresolved\n+fi\n+\n+# Check if the event is enabled\n+if ! grep -q 1 events/wprobes/my_wprobe/enable; then\n+ echo \"Failed to enable wprobe event\"\n+ exit_fail\n+fi\n+\n+# Let some time pass to trigger the breakpoint\n+sleep 1\n+\n+# Check if we got any trace output\n+if ! grep -q my_wprobe trace; then\n+ echo \"wprobe event was not triggered\"\n+ exit_fail\n+fi\n+\n+echo 0 \u003e events/wprobes/my_wprobe/enable\n+\n+# Check if the event is disabled\n+if ! grep -q 0 events/wprobes/my_wprobe/enable; then\n+ echo \"Failed to disable wprobe event\"\n+ exit_fail\n+fi\n+\n+echo \"-:my_wprobe\" \u003e\u003e dynamic_events\n+\n+if grep -q my_wprobe dynamic_events; then\n+ echo \"Failed to remove wprobe event\"\n+ exit_fail\n+fi\n+\n+if [ -d events/wprobes/my_wprobe ]; then\n+ echo \"Failed to remove wprobe event directory\"\n+ exit_fail\n+fi\n+\n+clear_trace\n+\n+exit 0\ndiff --git a/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc\nnew file mode 100644\nindex 0000000000000..e0df58fe7aa3e\n--- /dev/null\n+++ b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc\n@@ -0,0 +1,23 @@\n+#!/bin/sh\n+# SPDX-License-Identifier: GPL-2.0\n+# description: Watch probe event parser error log check\n+# requires: dynamic_events \"w[:[\u003cgroup\u003e/][\u003cevent\u003e]] [r|w|rw]@\u003caddr\u003e[:\u003clen\u003e]\":README\n+\n+check_error() { # command-with-error-pos-by-^\n+ ftrace_errlog_check 'wprobe' \"$1\" 'dynamic_events'\n+}\n+\n+check_error '^w'\t\t\t# NO_ARG_BODY\n+check_error 'w^10 w@jiffies'\t\t# WPROBE_NO_MAXACT\n+check_error 'w ^symbol'\t\t\t# BAD_ACCESS_FMT\n+check_error 'w ^a@symbol'\t\t# BAD_ACCESS_TYPE\n+check_error 'w w@^symbol'\t\t# BAD_ACCESS_ADDR\n+check_error 'w w@^0:4'\t\t\t# BAD_ACCESS_ADDR\n+check_error 'w w@jiffies^+offset'\t# BAD_PROBE_ADDR\n+check_error 'w w@jiffies:^100'\t\t# BAD_ACCESS_LEN\n+check_error 'w w@jiffies ^$arg1'\t# BAD_VAR\n+check_error 'w w@jiffies ^$retval'\t# BAD_VAR\n+check_error 'w w@jiffies ^$stack'\t# BAD_VAR\n+check_error 'w w@jiffies ^%ax'\t\t# BAD_VAR\n+\n+exit 0\ndiff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc\nnew file mode 100644\nindex 0000000000000..8bc8ca02a712e\n--- /dev/null\n+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc\n@@ -0,0 +1,85 @@\n+#!/bin/sh\n+# SPDX-License-Identifier: GPL-2.0\n+# description: event trigger - test set_wprobe trigger with BTF struct offset\n+# requires: dynamic_events \"w[:[\u003cgroup\u003e/][\u003cevent\u003e]] [r|w|rw]@\u003caddr\u003e[:\u003clen\u003e]\":README \"f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\":README events/sched/sched_process_fork/trigger \"[(structname[,field])]\u003cargname\u003e[-\u003efield[-\u003efield|.field...]]\":README\n+\n+fail() { #msg\n+ echo \"$1\"\n+ exit_fail\n+}\n+\n+rmmod trace-events-sample ||:\n+if ! modprobe trace-events-sample ; then\n+ echo \"No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m\"\n+ exit_unresolved\n+fi\n+\n+cleanup_wprobe_triggers() {\n+ if [ -f events/fprobes/testevent/trigger ]; then\n+ reset_trigger_file events/fprobes/testevent/trigger || true\n+ fi\n+ echo 0 \u003e events/enable 2\u003e/dev/null || true\n+ echo \u003e dynamic_events 2\u003e/dev/null || true\n+ sleep 1\n+ rmmod trace-events-sample 2\u003e/dev/null || true\n+ return 0\n+}\n+\n+trap cleanup_wprobe_triggers EXIT\n+\n+echo 0 \u003e tracing_on\n+\n+# we will skip this test if fprobe is not supported.\n+if ! grep -Fq \"f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\" README; then\n+ echo \"UNRESOLVED: fprobe is not supported\"\n+ exit_unresolved\n+fi\n+\n+# we will skip this test if the target function does not exist.\n+if ! grep -wq \"sample_timer_cb\" /proc/kallsyms; then\n+ echo \"UNRESOLVED: sample_timer_cb not found\"\n+ exit_unresolved\n+fi\n+\n+:;: \"Add a wprobe event watching 8 bytes\" ;:\n+echo 'w:watch rw@-1:8 address=$addr value=$value' \u003e\u003e dynamic_events\n+\n+:;: \"Add fprobe event for sample_timer_cb\" ;:\n+# sample_timer_cb(struct timer_list *t)\n+# container_of(t, struct foo_timer_data, timer)\n+echo 'f:fprobes/testevent sample_timer_cb timer=t' \u003e\u003e dynamic_events\n+\n+:;: \"Enable all events before setting triggers\" ;:\n+echo 1 \u003e tracing_on\n+echo 1 \u003e\u003e events/fprobes/testevent/enable\n+\n+:;: \"Set set_wprobe trigger using BTF struct offset resolution\" ;:\n+# Syntax: set_wprobe:WPEVENT:(STRUCT,FIELD)EVENT_FIELD-\u003eMEMBER\n+# (foo_timer_data,timer) is the BTF struct type and field name\n+# timer-\u003eexpires is the struct member whose offset is resolved automatically via BTF\n+echo 'set_wprobe:watch:(foo_timer_data,timer)timer-\u003etimer.expires' \u003e\u003e events/fprobes/testevent/trigger\n+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then\n+ fail \"Failed to set set_wprobe trigger\"\n+fi\n+\n+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate\n+sleep 3\n+\n+:;: \"Check set_wprobe trigger activated the watchpoint\" ;:\n+if ! grep -q watch trace; then\n+ fail \"Failed to trigger watchpoint\"\n+fi\n+\n+:;: \"Remove wprobe triggers\" ;:\n+# Since we don't know actual offset of timer-\u003eexpires in foo_timer_data, we use reset_trigger_file\n+reset_trigger_file events/fprobes/testevent/trigger\n+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then\n+ fail \"Failed to remove set_wprobe trigger\"\n+fi\n+\n+:;: \"Disable events and remove dynamic events\" ;:\n+echo 0 \u003e events/enable\n+echo \u003e dynamic_events\n+clear_trace\n+\n+exit 0\ndiff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc\nnew file mode 100644\nindex 0000000000000..3d4c9fd6b8e23\n--- /dev/null\n+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc\n@@ -0,0 +1,85 @@\n+#!/bin/sh\n+# SPDX-License-Identifier: GPL-2.0\n+# description: event trigger - test wprobe trigger with BTF typecast fetchargs\n+# requires: dynamic_events \"w[:[\u003cgroup\u003e/][\u003cevent\u003e]] [r|w|rw]@\u003caddr\u003e[:\u003clen\u003e]\":README \"f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\":README events/sched/sched_process_fork/trigger \"[(structname[,field])]\u003cargname\u003e[-\u003efield[-\u003efield|.field...]]\":README\n+\n+fail() { #msg\n+ echo \"$1\"\n+ exit_fail\n+}\n+\n+rmmod trace-events-sample ||:\n+if ! modprobe trace-events-sample ; then\n+ echo \"No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m\"\n+ exit_unresolved\n+fi\n+\n+cleanup_wprobe_triggers() {\n+ if [ -f events/fprobes/testevent/trigger ]; then\n+ reset_trigger_file events/fprobes/testevent/trigger || true\n+ fi\n+ echo 0 \u003e events/enable 2\u003e/dev/null || true\n+ echo \u003e dynamic_events 2\u003e/dev/null || true\n+ sleep 1\n+ rmmod trace-events-sample 2\u003e/dev/null || true\n+ return 0\n+}\n+\n+trap cleanup_wprobe_triggers EXIT\n+\n+echo 0 \u003e tracing_on\n+\n+# we will skip this test if fprobe is not supported.\n+if ! grep -Fq \"f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\" README; then\n+ echo \"UNRESOLVED: fprobe is not supported\"\n+ exit_unresolved\n+fi\n+\n+# we will skip this test if the target function does not exist.\n+if ! grep -wq \"sample_timer_cb\" /proc/kallsyms; then\n+ echo \"UNRESOLVED: sample_timer_cb not found\"\n+ exit_unresolved\n+fi\n+\n+:;: \"Add a wprobe event with BTF typecast fetchargs\" ;:\n+# (foo_timer_data,timer)$addr-\u003ecounter reads counter from struct foo_timer_data via BTF typecast\n+echo 'w:watch rw@-1:8 address=$addr counter=(foo_timer_data,timer)$addr-\u003ecounter' \u003e\u003e dynamic_events\n+\n+:;: \"Check the wprobe event is registered with counter field\" ;:\n+if ! grep -q \"counter\" dynamic_events; then\n+ fail \"Failed to register counter field in dynamic_events\"\n+fi\n+\n+:;: \"Add fprobe event for sample_timer_cb\" ;:\n+echo 'f:fprobes/testevent sample_timer_cb timer=t' \u003e\u003e dynamic_events\n+\n+:;: \"Enable all events before setting triggers\" ;:\n+echo 1 \u003e tracing_on\n+echo 1 \u003e\u003e events/fprobes/testevent/enable\n+\n+:;: \"Set set_wprobe trigger on testevent\" ;:\n+echo 'set_wprobe:watch:timer' \u003e\u003e events/fprobes/testevent/trigger\n+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then\n+ fail \"Failed to set set_wprobe trigger\"\n+fi\n+\n+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate\n+sleep 3\n+\n+:;: \"Check set_wprobe trigger activated the watchpoint\" ;:\n+if ! grep -q watch trace; then\n+ fail \"Failed to trigger watchpoint\"\n+fi\n+\n+:;: \"Remove wprobe triggers\" ;:\n+echo '!set_wprobe:watch:timer' \u003e\u003e events/fprobes/testevent/trigger\n+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then\n+ fail \"Failed to remove set_wprobe trigger\"\n+fi\n+\n+:;: \"Disable events and remove dynamic events\" ;:\n+echo 0 \u003e events/enable\n+echo \u003e dynamic_events\n+clear_trace\n+\n+exit 0\ndiff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc\nnew file mode 100644\nindex 0000000000000..f82b13d813db1\n--- /dev/null\n+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc\n@@ -0,0 +1,150 @@\n+#!/bin/sh\n+# SPDX-License-Identifier: GPL-2.0\n+# description: event trigger - test wprobe trigger across cpu hotplug\n+# requires: dynamic_events \"w[:[\u003cgroup\u003e/][\u003cevent\u003e]] [r|w|rw]@\u003caddr\u003e[:\u003clen\u003e]\":README \"f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\":README events/sched/sched_process_fork/trigger \"[(structname[,field])]\u003cargname\u003e[-\u003efield[-\u003efield|.field...]]\":README\n+\n+fail() { #msg\n+ echo \"$1\"\n+ exit_fail\n+}\n+\n+if ! which nproc \u003e/dev/null 2\u003e\u00261 ; then\n+ nproc() {\n+ ls -d /sys/devices/system/cpu/cpu[0-9]* | wc -l\n+ }\n+fi\n+\n+NP=`nproc`\n+if [ $NP -le 1 ] ; then\n+ echo \"We cannot test cpu hotplug in UP environment\"\n+ exit_unresolved\n+fi\n+\n+# Find an online CPU that can be offlined\n+for i in /sys/devices/system/cpu/cpu[1-9]*; do\n+ if [ -f $i/online ] \u0026\u0026 [ \"$(cat $i/online)\" = \"1\" ]; then\n+ cpu=$i\n+ break\n+ fi\n+done\n+\n+if [ -z \"$cpu\" ]; then\n+ echo \"We cannot test cpu hotplug without a hotpluggable online cpu\"\n+ exit_unresolved\n+fi\n+\n+rmmod trace-events-sample ||:\n+if ! modprobe trace-events-sample ; then\n+ echo \"No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m\"\n+ exit_unresolved\n+fi\n+\n+cleanup_wprobe_triggers() {\n+ [ -n \"$cpu\" ] \u0026\u0026 [ -f \"$cpu/online\" ] \u0026\u0026 echo 1 \u003e $cpu/online || true\n+ if [ -f events/fprobes/testevent/trigger ]; then\n+ reset_trigger_file events/fprobes/testevent/trigger || true\n+ fi\n+ if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then\n+ reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true\n+ fi\n+ echo 0 \u003e events/enable 2\u003e/dev/null || true\n+ echo \u003e dynamic_events 2\u003e/dev/null || true\n+ sleep 1\n+ rmmod trace-events-sample 2\u003e/dev/null || true\n+ return 0\n+}\n+\n+trap cleanup_wprobe_triggers EXIT\n+\n+echo 0 \u003e tracing_on\n+\n+# we will skip this test if fprobe is not supported.\n+if ! grep -Fq \"f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\" README; then\n+ echo \"UNRESOLVED: fprobe is not supported\"\n+ exit_unresolved\n+fi\n+\n+# we will skip this test if the target function does not exist.\n+if ! grep -wq \"sample_timer_cb\" /proc/kallsyms; then\n+ echo \"UNRESOLVED: sample_timer_cb not found\"\n+ exit_unresolved\n+fi\n+\n+:;: \"Add a wprobe event used by trigger\" ;:\n+echo 'w:watch rw@-1:8 address=$addr value=$value' \u003e dynamic_events\n+\n+:;: \"Add events for triggering wprobe\" ;:\n+echo 'f:fprobes/testevent sample_timer_cb timer=t' \u003e\u003e dynamic_events\n+\n+:;: \"Enable all events before setting triggers\" ;:\n+echo 1 \u003e tracing_on\n+echo 1 \u003e\u003e events/fprobes/testevent/enable\n+echo 1 \u003e\u003e events/sample-trace/foo_bar_with_fn/enable\n+\n+:;: \"Set set_wprobe trigger on testevent\" ;:\n+echo 'set_wprobe:watch:timer' \u003e\u003e events/fprobes/testevent/trigger\n+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then\n+ fail \"Failed to set set_wprobe trigger\"\n+fi\n+\n+:;: \"Take target CPU offline\" ;:\n+echo 0 \u003e $cpu/online\n+sleep 1\n+\n+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate while CPU is offline\n+sleep 2\n+\n+:;: \"Check set_wprobe trigger activated the watchpoint\" ;:\n+if ! grep -q watch trace; then\n+ fail \"Failed to trigger watchpoint while CPU is offline\"\n+fi\n+\n+:;: \"Bring target CPU back online\" ;:\n+echo 1 \u003e $cpu/online\n+sleep 1\n+\n+# Clear trace and verify that watchpoint triggers after CPU is onlined\n+clear_trace\n+sleep 2\n+\n+:;: \"Verify watchpoint triggers after CPU came back online\" ;:\n+if ! grep -q watch trace; then\n+ fail \"Failed to trigger watchpoint after CPU was onlined\"\n+fi\n+\n+:;: \"Set clear_wprobe trigger on foo_bar_with_fn\" ;:\n+echo 'clear_wprobe:watch' \u003e\u003e events/sample-trace/foo_bar_with_fn/trigger\n+if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then\n+ fail \"Failed to set clear_wprobe trigger\"\n+fi\n+\n+# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpoint\n+echo '!set_wprobe:watch:timer' \u003e\u003e events/fprobes/testevent/trigger\n+\n+# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate watchpoint\n+sleep 2\n+\n+# Clear trace and wait to ensure no new watchpoint events are generated\n+clear_trace\n+sleep 1\n+\n+:;: \"Ensure clear_wprobe trigger deactivated the watchpoint\" ;:\n+if grep -q watch trace; then\n+ fail \"Failed to clear watchpoint\"\n+fi\n+\n+:;: \"Remove wprobe triggers\" ;:\n+echo '!clear_wprobe:watch' \u003e\u003e events/sample-trace/foo_bar_with_fn/trigger\n+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then\n+ fail \"Failed to remove set_wprobe trigger\"\n+fi\n+if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then\n+ fail \"Failed to remove clear_wprobe trigger\"\n+fi\n+\n+:;: \"Disable events and remove dynamic events\" ;:\n+echo 0 \u003e events/enable\n+echo \u003e dynamic_events\n+clear_trace\n+\n+exit 0\ndiff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc\nnew file mode 100644\nindex 0000000000000..dd1ce4baa62be\n--- /dev/null\n+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc\n@@ -0,0 +1,37 @@\n+#!/bin/sh\n+# SPDX-License-Identifier: GPL-2.0\n+# description: event trigger - test wprobe trigger syntax errors\n+# requires: dynamic_events error_log \"w[:[\u003cgroup\u003e/][\u003cevent\u003e]] [r|w|rw]@\u003caddr\u003e[:\u003clen\u003e]\":README \"f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\":README events/sched/sched_process_fork/trigger\n+\n+check_error() { # command-with-error-pos-by-^\n+ ftrace_errlog_check \"wprobe_trigger\" \"$1\" \"events/fprobes/testevent/trigger\"\n+}\n+\n+TARGET_FUNC=$(grep -m 1 -E -w \"[tT] (vfs_read|do_sys_openat2|do_sys_open)\" /proc/kallsyms | awk '{print $3}')\n+if [ -z \"$TARGET_FUNC\" ]; then\n+ echo \"UNRESOLVED: target function not found\"\n+ exit_unresolved\n+fi\n+\n+# Add a dummy fprobe event to attach triggers to\n+echo \"f:fprobes/testevent $TARGET_FUNC dfd=\\$arg1\" \u003e dynamic_events\n+\n+# Add a target wprobe event\n+echo 'w:watch rw@-1:8' \u003e\u003e dynamic_events\n+\n+# Test errors on trigger syntax\n+check_error 'set_wprobe:^non_exist_wprobe:dfd'\t# WPROBE_NOT_FOUND\n+check_error 'set_wprobe:^'\t\t\t\t# WPROBE_NOT_FOUND\n+check_error 'set_wprobe:watch^'\t\t\t\t# WPROBE_NEED_FIELD\n+check_error 'set_wprobe:watch:^non_exist_field'\t# NO_EVENT_FIELD\n+\n+# Enable target wprobe event and test WPROBE_BUSY error\n+echo 1 \u003e events/wprobes/watch/enable\n+check_error 'set_wprobe:^watch:dfd'\t\t\t# WPROBE_BUSY\n+echo 0 \u003e events/wprobes/watch/enable\n+\n+# Cleanup\n+echo '-:watch' \u003e\u003e dynamic_events\n+echo '-:testevent' \u003e\u003e dynamic_events\n+\n+exit 0\ndiff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc\nnew file mode 100644\nindex 0000000000000..043e1c915ccf1\n--- /dev/null\n+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc\n@@ -0,0 +1,107 @@\n+#!/bin/sh\n+# SPDX-License-Identifier: GPL-2.0\n+# description: event trigger - test wprobe trigger\n+# requires: dynamic_events \"w[:[\u003cgroup\u003e/][\u003cevent\u003e]] [r|w|rw]@\u003caddr\u003e[:\u003clen\u003e]\":README \"f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\":README events/sched/sched_process_fork/trigger \"[(structname[,field])]\u003cargname\u003e[-\u003efield[-\u003efield|.field...]]\":README\n+\n+fail() { #msg\n+ echo \"$1\"\n+ exit_fail\n+}\n+\n+rmmod trace-events-sample ||:\n+if ! modprobe trace-events-sample ; then\n+ echo \"No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m\"\n+ exit_unresolved\n+fi\n+\n+cleanup_wprobe_triggers() {\n+ if [ -f events/fprobes/testevent/trigger ]; then\n+ reset_trigger_file events/fprobes/testevent/trigger || true\n+ fi\n+ if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then\n+ reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true\n+ fi\n+ echo 0 \u003e events/enable 2\u003e/dev/null || true\n+ echo \u003e dynamic_events 2\u003e/dev/null || true\n+ sleep 1\n+ rmmod trace-events-sample 2\u003e/dev/null || true\n+ return 0\n+}\n+\n+trap cleanup_wprobe_triggers EXIT\n+\n+echo 0 \u003e tracing_on\n+\n+# we will skip this test if fprobe is not supported.\n+if ! grep -Fq \"f[:[\u003cgroup\u003e/][\u003cevent\u003e]] \u003cfunc-name\u003e[%return] [\u003cargs\u003e]\" README; then\n+ echo \"UNRESOLVED: fprobe is not supported\"\n+ exit_unresolved\n+fi\n+\n+# we will skip this test if the target function does not exist.\n+if ! grep -wq \"sample_timer_cb\" /proc/kallsyms; then\n+ echo \"UNRESOLVED: sample_timer_cb not found\"\n+ exit_unresolved\n+fi\n+\n+:;: \"Add a wprobe event used by trigger\" ;:\n+echo 'w:watch rw@-1:8 address=$addr value=$value' \u003e dynamic_events\n+\n+:;: \"Add events for triggering wprobe\" ;:\n+echo 'f:fprobes/testevent sample_timer_cb timer=t' \u003e\u003e dynamic_events\n+\n+:;: \"Enable all events before setting triggers\" ;:\n+echo 1 \u003e tracing_on\n+echo 1 \u003e\u003e events/fprobes/testevent/enable\n+echo 1 \u003e\u003e events/sample-trace/foo_bar_with_fn/enable\n+\n+:;: \"Set set_wprobe trigger on testevent\" ;:\n+echo 'set_wprobe:watch:timer' \u003e\u003e events/fprobes/testevent/trigger\n+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then\n+ fail \"Failed to set set_wprobe trigger\"\n+fi\n+\n+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate\n+sleep 2\n+\n+:;: \"Check set_wprobe trigger activated the watchpoint\" ;:\n+if ! grep -q watch trace; then\n+ fail \"Failed to trigger watchpoint\"\n+fi\n+\n+:;: \"Set clear_wprobe trigger on foo_bar_with_fn\" ;:\n+echo 'clear_wprobe:watch' \u003e\u003e events/sample-trace/foo_bar_with_fn/trigger\n+if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then\n+ fail \"Failed to set clear_wprobe trigger\"\n+fi\n+\n+# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpoint\n+echo '!set_wprobe:watch:timer' \u003e\u003e events/fprobes/testevent/trigger\n+\n+# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate watchpoint\n+sleep 2\n+\n+# Clear trace and wait to ensure no new watchpoint events are generated\n+clear_trace\n+sleep 1\n+\n+:;: \"Ensure clear_wprobe trigger deactivated the watchpoint\" ;:\n+if grep -q watch trace; then\n+ fail \"Failed to clear watchpoint\"\n+fi\n+\n+:;: \"Remove wprobe triggers\" ;:\n+echo '!clear_wprobe:watch' \u003e\u003e events/sample-trace/foo_bar_with_fn/trigger\n+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then\n+ fail \"Failed to remove set_wprobe trigger\"\n+fi\n+if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then\n+ fail \"Failed to remove clear_wprobe trigger\"\n+fi\n+\n+:;: \"Disable events and remove dynamic events\" ;:\n+echo 0 \u003e events/enable\n+echo \u003e dynamic_events\n+clear_trace\n+\n+exit 0\ndiff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c\nindex 65eb26a0520d8..7779d500fe774 100644\n--- a/virt/kvm/kvm_main.c\n+++ b/virt/kvm/kvm_main.c\n@@ -6476,11 +6476,23 @@ static unsigned long kvm_guest_get_ip(void)\n \treturn kvm_arch_vcpu_get_ip(vcpu);\n }\n \n+static bool kvm_guest_in_guest(void)\n+{\n+\tstruct kvm_vcpu *vcpu = kvm_get_running_vcpu();\n+\n+\tif (!vcpu)\n+\t\treturn false;\n+\n+\treturn READ_ONCE(vcpu-\u003emode) == IN_GUEST_MODE ||\n+\t READ_ONCE(vcpu-\u003emode) == EXITING_GUEST_MODE;\n+}\n+\n static struct perf_guest_info_callbacks kvm_guest_cbs = {\n \t.state\t\t\t= kvm_guest_state,\n \t.get_ip\t\t\t= kvm_guest_get_ip,\n \t.handle_intel_pt_intr\t= NULL,\n \t.handle_mediated_pmi\t= NULL,\n+\t.in_guest\t\t= kvm_guest_in_guest,\n };\n \n void __kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void),\n"
} |
| 2/1 | 2026/09/24 05:44 | agent | patch-evaluator |
0mModel:core Args: null Results: {
"EnableConfigs": [
"WPROBE_EVENTS",
"WPROBE_TRIGGERS",
"PROBE_EVENTS_BTF_ARGS"
],
"FocusSymbols": [
"trace_wprobe_create",
"wprobe_perf_handler",
"wprobe_trace_handler",
"wprobe_trigger",
"wprobe_trigger_cmd_parse",
"wprobe_work_func",
"modify_local_hw_breakpoint_addr",
"arch_modify_local_hw_breakpoint_addr"
],
"Reasoning": "The patch introduces watchpoint probe (wprobe) dynamic event tracing and triggers (trace_wprobe.c), allowing userspace to configure hardware watchpoints on kernel memory addresses through the tracefs dynamic_events and trigger interfaces. It also adds new hardware breakpoint manipulation infrastructure (modify_local_hw_breakpoint_addr, arch_modify_local_hw_breakpoint_addr) and modifies x86 DR7 register synchronization across traps, NMIs, and MCEs. These changes introduce significant new parsing, concurrency, and hardware breakpoint handling logic directly exposed to userspace fuzzing.",
"WorthFuzzing": true
}Instruction: You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.
IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.
================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.
- REACHABILITY IS THE PRIMARY GATE:
Fuzzing can only discover bugs in code that can actually execute in standard virtualized
environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
regardless of whether it adds assertions or complex logic.
- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
Patch authors routinely label changes as "cleanups", "refactorings", or state
"No functional change intended". Do NOT take these claims at face value.
Code refactorings that rearrange logic, introduce helper functions, or alter state management
in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
If reachable executable code is modified or refactored, it MUST be fuzzed.
- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
Even if the author believes the invariant always holds, fuzzing is essential to verify whether
an unusual sequence of operations can violate it.
================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:
- Non-kernel and non-executable changes:
* Modifications to Documentation/, comments, or spelling fixes.
* User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
that do not affect the compiled kernel image (vmlinux) or kernel modules.
* Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
that do not alter control flow or data structures.
* Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
* Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
* Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
physical PCI hot-unplug or manual sysfs driver unbinding.
* Code paths exclusive to architectures other than the target architecture.
================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
* Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
* Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
* Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
* Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).
================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:
- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
Do NOT list generic, ubiquitous functions called by almost every program in the corpus
(including, but not limited to: general memory allocators and deallocators, page fault
and trap handlers, or core synchronization primitives; this is not an exhaustive list).
Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
which severely dilutes fuzzing effort away from the actual changes.
- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
List functions that specifically implement the logic being added or altered, or direct API entrypoints
for the subsystem feature under review.
- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).
================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit aa59edb7da5ec405cf1653e89aee6e32a2266101
Author: syz-cluster <triage@syzkaller.com>
Date: Thu Sep 24 05:44:54 2026 +0000
syz-cluster: applied patch under review
diff --git a/Documentation/trace/index.rst b/Documentation/trace/index.rst
index f4058e8e92e3d..4f3fa60e4ecd9 100644
--- a/Documentation/trace/index.rst
+++ b/Documentation/trace/index.rst
@@ -36,6 +36,7 @@ the Linux kernel.
kprobes
kprobetrace
fprobetrace
+ wprobetrace
eprobetrace
fprobe
ring-buffer-design
diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wprobetrace.rst
new file mode 100644
index 0000000000000..1d9b32d02d53b
--- /dev/null
+++ b/Documentation/trace/wprobetrace.rst
@@ -0,0 +1,200 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================================
+Watchpoint probe (wprobe) Event Tracing
+=======================================
+
+.. Author: Masami Hiramatsu <mhiramat@kernel.org>
+
+Overview
+--------
+
+Wprobe event is a dynamic event based on the hardware breakpoint, which is
+similar to other probe events, but it is for watching data access. It allows
+you to trace which code accesses a specified data.
+
+As same as other dynamic events, wprobe events are defined via
+`dynamic_events` interface file on tracefs.
+
+Synopsis of wprobe-events
+-------------------------
+::
+
+ w:[GRP/][EVENT] SPEC [FETCHARGS] : Probe on data access
+
+ GRP : Group name for wprobe. If omitted, use "wprobes" for it.
+ EVENT : Event name for wprobe. If omitted, an event name is
+ generated based on the address or symbol.
+ SPEC : Breakpoint specification.
+ [r|w|rw]@<ADDRESS|SYMBOL[+|-OFFS]>[:LENGTH]
+
+ r|w|rw : Access type, r for read, w for write, and rw for both.
+ Default is rw if omitted.
+ ADDRESS : Address to watch (hexadecimal). MUST be in kernel space.
+ SYMBOL[+|-OFFS] : Symbol name to watch. (Optional positive/negative offset)
+ LENGTH : Length of the data to watch in bytes. (1, 2, 4, or 8)
+ Default is 4.
+
+ FETCHARGS : Arguments. Each probe can have up to 128 args.
+ $addr : Fetch the accessing address.
+ $value : Fetch the memory value at the accessing address (same as +0($addr)).
+ @ADDR : Fetch memory at ADDR (ADDR should be in kernel)
+ @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol)
+ +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*1)(\*2)
+ \IMM : Store an immediate value to the argument.
+ NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
+ FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types
+ (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
+ (x8/x16/x32/x64), "char", "string", "ustring", "symbol", "symstr"
+ and bitfield are supported.
+ (STRUCT[,ASGN])FETCHARG->MEMBER[->MEMBER] : If BTF is supported, typecast
+ FETCHARG to a pointer to STRUCT and then dereference the
+ pointer defined by ->MEMBER. ASGN can be specified optionally.
+ If ASGN is specified, FETCHARG will be cast to the same offset
+ position as the ASGN member, rather than to the beginning of
+ the STRUCT.
+ (STRUCT[,ASGN])(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above
+ can also be used with another FETCHARG.
+
+ (\*1) This is useful for fetching a field of data structures.
+ (\*2) "u" means user-space dereference.
+
+For the details of TYPE, see :ref:`kprobetrace documentation <kprobetrace_types>`.
+
+Usage examples
+--------------
+Here is an example to add a wprobe event on a variable `jiffies`.
+::
+
+ # echo 'w:my_jiffies w@jiffies' >> dynamic_events
+ # cat dynamic_events
+ w:wprobes/my_jiffies w@jiffies
+ # echo 1 > events/wprobes/enable
+ # cat trace | head
+ # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
+ # | | | ||||| | |
+ <idle>-0 [000] d.Z1. 717.026259: my_jiffies: (tick_do_update_jiffies64+0xbe/0x130)
+ <idle>-0 [000] d.Z1. 717.026373: my_jiffies: (tick_do_update_jiffies64+0xbe/0x130)
+
+You can see the code which writes to `jiffies` is `tick_do_update_jiffies64()`.
+
+Notes
+-----
+Wprobe event does not disable itself even if the module is unloaded.
+For example, if you add a wprobe event on a module variable, and then
+unload the module, the wprobe event will still be enabled. This is for
+watching the address is used unexpectedly after the module is unloaded.
+
+Combination with trigger action
+-------------------------------
+The event trigger action can extend the utilization of this wprobe.
+
+- set_wprobe:WPEVENT:FIELD[+|-ADJUST][:COUNT]
+- set_wprobe:WPEVENT:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COUNT]
+- clear_wprobe:WPEVENT[:FIELD[+|-ADJUST][:COUNT]]
+- clear_wprobe:WPEVENT[:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COUNT]]
+
+Set these triggers to the target event, then the WPROBE event will be
+setup to trace the memory access at FIELD[+|-ADJUST] address.
+If BTF is supported, the target address can also be resolved using BTF
+struct offset resolution: `(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST]`.
+Here, EVENT_FIELD is an event field containing a pointer to STRUCT (or to
+the ASGN member if ASGN is specified), and MEMBER is the struct member whose
+offset is resolved automatically via BTF.
+When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is
+forcibly cleared. If FIELD[+|-ADJUST] is set, it clears WPEVENT only
+if its watching address is the same as the FIELD[+|-ADJUST] value.
+If COUNT is specified, it will set/clear WPEVENT only if it hits COUNT
+times.
+
+Notes:
+- set_wprobe only works on the wprobe which is NOT set a valid address yet,
+ and it must be enabled after the trigger is set.
+- clear_wprobe only works on the wprobe which is set a valid address, and it
+ will be soft-disabled after the trigger is cleared.
+- Therefore, if a trigger sets/clears a wprobe, other/same trigger events
+ will not work (on the same event) while the wprobe is set.
+
+The set_wprobe trigger does not change the type and length, these
+must be set when creating a new wprobe.
+
+The WPROBE event must be disabled when setting the new trigger
+and it will be busy afterwards. Recommended usage is to add a new
+wprobe at invalid dummy address (-1) and keep disabled.
+
+Wprobe triggers only support target addresses in kernel memory. If a
+set_wprobe trigger evaluates to a user-space memory address or NULL
+pointer, the trigger action ignores the update and skips setting the
+watchpoint.
+
+Wprobe triggers are not supported on kprobe_events, because kprobes
+themselves can use software breakpoints which conflicts with wprobe
+operation.
+
+
+For example, trace the first 8 bytes of the dentry data structure passed
+to do_truncate() until it is deleted by dentry_kill().
+(Note: all tracefs setup uses '>>' so that it does not kick do_truncate())
+::
+
+ # echo 'w:watch rw@-1:8 address=$addr value=+0($addr)' >> dynamic_events
+ # echo 'f:truncate do_truncate dentry=$arg2' >> dynamic_events
+ # echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger
+ # echo 'f:dentry_kill dentry_kill dentry=$arg1' >> dynamic_events
+ # echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger
+ # echo 1 >> events/fprobes/truncate/enable
+ # echo 1 >> events/fprobes/dentry_kill/enable
+
+ # echo aaa > /tmp/hoge
+ # echo bbb > /tmp/hoge
+ # echo ccc > /tmp/hoge
+ # rm /tmp/hoge
+
+Then, the trace data will show::
+
+ # tracer: nop
+ #
+ # entries-in-buffer/entries-written: 32/32 #P:8
+ #
+ # _-----=> irqs-off/BH-disabled
+ # / _----=> need-resched
+ # | / _---=> hardirq/softirq
+ # || / _--=> preempt-depth
+ # ||| / _-=> migrate-disable
+ # |||| / delay
+ # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
+ # | | | ||||| | |
+ sh-107 [004] ...1. 9.990418: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+ sh-107 [004] ...1. 9.990914: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b3de78
+ sh-107 [004] ...1. 9.993175: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049ddd40
+ sh-107 [004] ..... 9.995198: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 9.995389: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db998
+ sh-107 [004] ..Zff 9.997503: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997509: watch: (path_openat+0x211/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997514: watch: (path_openat+0xa56/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997518: watch: (path_openat+0xae2/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..... 9.997521: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 9.997582: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004808270
+ sh-107 [004] ...1. 9.999365: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db728
+ sh-107 [004] ...1. 9.999388: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b1c000
+ rm-113 [005] ..Zff 10.000965: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000971: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000984: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000988: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001010: watch: (lookup_one_qstr_excl+0x28/0x140) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001014: watch: (lookup_one_qstr_excl+0xd1/0x140) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001018: watch: (may_delete_dentry+0x1c/0x200) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001021: watch: (may_delete_dentry+0x195/0x200) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001031: watch: (vfs_unlink+0x5e/0x260) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] d.Z.. 10.001067: watch: (d_make_discardable+0x1b/0x40) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] d.Z.. 10.001071: watch: (d_make_discardable+0x29/0x40) address=0xffff8880048083a8 value=0x200080
+ rm-113 [005] ...1. 10.001072: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+ rm-113 [005] ...1. 10.001218: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 10.001416: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db110
+ sh-107 [004] ...1. 10.001444: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db248
+ sh-107 [004] ...1. 10.001500: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+ sh-107 [004] ...1. 10.002067: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+ sh-107 [004] ...1. 10.904920: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+ sh-107 [004] ...1. 10.905129: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+
+You can see the watch event is correctly configured on the dentry.
diff --git a/arch/Kconfig b/arch/Kconfig
index 45c6577723623..5292bd05d5d4d 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -474,6 +474,24 @@ config HAVE_MIXED_BREAKPOINTS_REGS
Select this option if your arch implements breakpoints under the
latter fashion.
+config HAVE_POST_BREAKPOINT_HOOK
+ bool
+ depends on HAVE_HW_BREAKPOINT
+ help
+ Depending on the arch implementation of hardware breakpoints,
+ some of them provide breakpoint hook after the target memory
+ is modified.
+ Select this option if your arch implements breakpoints overflow
+ handler hooks after the target memory is modified.
+
+config HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
+ bool
+ depends on HAVE_HW_BREAKPOINT
+ help
+ Select this if the architecture can modify the address of an
+ installed hardware breakpoint on the local CPU without releasing
+ and reserving its hardware slot.
+
config HAVE_USER_RETURN_NOTIFIER
bool
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 15fd9ec5ecacb..e4eaf0b607d30 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -249,6 +249,7 @@ config X86
select HAVE_FUNCTION_TRACER
select HAVE_GCC_PLUGINS
select HAVE_HW_BREAKPOINT
+ select HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
select HAVE_IOREMAP_PROT
select HAVE_IRQ_EXIT_ON_IRQ_STACK if X86_64
select HAVE_IRQ_TIME_ACCOUNTING
@@ -288,6 +289,7 @@ config X86
select MMU_GATHER_RCU_TABLE_FREE
select MMU_GATHER_MERGE_VMAS
select HAVE_POSIX_CPU_TIMERS_TASK_WORK
+ select HAVE_POST_BREAKPOINT_HOOK
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_RELIABLE_STACKTRACE if UNWINDER_ORC || STACK_VALIDATION
select HAVE_FUNCTION_ARG_ACCESS_API
diff --git a/arch/x86/include/asm/debugreg.h b/arch/x86/include/asm/debugreg.h
index 854d82b88ff4e..515d2d313d0c8 100644
--- a/arch/x86/include/asm/debugreg.h
+++ b/arch/x86/include/asm/debugreg.h
@@ -18,6 +18,7 @@
#define DR7_FIXED_1 0x00000400
DECLARE_PER_CPU(unsigned long, cpu_dr7);
+DECLARE_PER_CPU(unsigned int, cpu_dr7_seq);
#ifndef CONFIG_PARAVIRT_XXL
/*
@@ -125,40 +126,69 @@ static __always_inline bool hw_breakpoint_active(void)
extern void hw_breakpoint_restore(void);
-static __always_inline unsigned long local_db_save(void)
+static __always_inline void local_db_save(unsigned long *dr7,
+ unsigned int *dr7_seq)
{
- unsigned long dr7;
+ bool retrying = false;
- if (cpu_feature_enabled(X86_FEATURE_HYPERVISOR) && !hw_breakpoint_active())
- return 0;
+ do {
+ /* Inner loop: get DR7 with checking cpu_dr7_seq. */
+ *dr7_seq = this_cpu_read(cpu_dr7_seq);
- get_debugreg(dr7, 7);
+ if (unlikely(retrying)) {
+ *dr7 = this_cpu_read(cpu_dr7);
+ } else {
+ get_debugreg(*dr7, 7);
- /* Architecturally set bit */
- dr7 &= ~DR7_FIXED_1;
- if (dr7)
- set_debugreg(DR7_FIXED_1, 7);
+ /* Architecturally set bit */
+ *dr7 &= ~DR7_FIXED_1;
+ }
- /*
- * Ensure the compiler doesn't lower the above statements into
- * the critical section; disabling breakpoints late would not
- * be good.
- */
- barrier();
+ barrier();
+ if (unlikely(*dr7_seq != this_cpu_read(cpu_dr7_seq))) {
+ retrying = true;
+ continue;
+ }
+
+ /* Outer loop: clear DR7 and retry if sequence number is updated. */
+ if (*dr7)
+ set_debugreg(DR7_FIXED_1, 7);
- return dr7;
+ /*
+ * Ensure the compiler doesn't lower the above statements into
+ * the critical section; disabling breakpoints late would not
+ * be good.
+ */
+ barrier();
+ retrying = true;
+ } while (unlikely(*dr7_seq != this_cpu_read(cpu_dr7_seq)));
}
-static __always_inline void local_db_restore(unsigned long dr7)
+static __always_inline void local_db_restore(unsigned long dr7,
+ unsigned int dr7_seq)
{
+ unsigned long val;
+ unsigned int seq;
+
/*
* Ensure the compiler doesn't raise this statement into
* the critical section; enabling breakpoints early would
* not be good.
*/
barrier();
- if (dr7)
- set_debugreg(dr7, 7);
+ do {
+ seq = this_cpu_read(cpu_dr7_seq);
+ if (seq == dr7_seq) {
+ if (!dr7)
+ return;
+ val = dr7;
+ } else {
+ val = this_cpu_read(cpu_dr7);
+ }
+
+ set_debugreg(val | DR7_FIXED_1, 7);
+ barrier();
+ } while (unlikely(seq != this_cpu_read(cpu_dr7_seq)));
}
#ifdef CONFIG_CPU_SUP_AMD
diff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw_breakpoint.h
index 0bc931cd0698b..f35ec491f6dda 100644
--- a/arch/x86/include/asm/hw_breakpoint.h
+++ b/arch/x86/include/asm/hw_breakpoint.h
@@ -59,6 +59,8 @@ extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
int arch_install_hw_breakpoint(struct perf_event *bp);
+int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr);
void arch_uninstall_hw_breakpoint(struct perf_event *bp);
void hw_breakpoint_pmu_read(struct perf_event *bp);
void hw_breakpoint_pmu_unthrottle(struct perf_event *bp);
diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
index ab469605fc893..b3669d6f88d4f 100644
--- a/arch/x86/kernel/cpu/mce/core.c
+++ b/arch/x86/kernel/cpu/mce/core.c
@@ -2108,6 +2108,10 @@ bool filter_mce(struct mce *m)
static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)
{
irqentry_state_t irq_state;
+ unsigned long dr7;
+ unsigned int dr7_seq;
+
+ local_db_save(&dr7, &dr7_seq);
WARN_ON_ONCE(user_mode(regs));
@@ -2116,20 +2120,27 @@ static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)
* mce_check_crashing_cpu() for details.
*/
if (mca_cfg.initialized && mce_check_crashing_cpu())
- return;
+ goto out;
irq_state = irqentry_nmi_enter(regs);
do_machine_check(regs);
irqentry_nmi_exit(regs, irq_state);
+out:
+ local_db_restore(dr7, dr7_seq);
}
static __always_inline void exc_machine_check_user(struct pt_regs *regs)
{
+ unsigned long dr7;
+ unsigned int dr7_seq;
+
irqentry_enter_from_user_mode(regs);
+ local_db_save(&dr7, &dr7_seq);
do_machine_check(regs);
+ local_db_restore(dr7, dr7_seq);
irqentry_exit_to_user_mode(regs);
}
@@ -2138,21 +2149,13 @@ static __always_inline void exc_machine_check_user(struct pt_regs *regs)
/* MCE hit kernel mode */
DEFINE_IDTENTRY_MCE(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
exc_machine_check_kernel(regs);
- local_db_restore(dr7);
}
/* The user mode variant. */
DEFINE_IDTENTRY_MCE_USER(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
exc_machine_check_user(regs);
- local_db_restore(dr7);
}
#ifdef CONFIG_X86_FRED
@@ -2169,28 +2172,20 @@ DEFINE_IDTENTRY_MCE_USER(exc_machine_check)
*/
DEFINE_FREDENTRY_MCE(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
if (user_mode(regs))
exc_machine_check_user(regs);
else
exc_machine_check_kernel(regs);
- local_db_restore(dr7);
}
#endif
#else
/* 32bit unified entry point */
DEFINE_IDTENTRY_RAW(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
if (user_mode(regs))
exc_machine_check_user(regs);
else
exc_machine_check_kernel(regs);
- local_db_restore(dr7);
}
#endif
diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index f846c15f21ca1..b413adc2bbbf4 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -40,6 +40,8 @@
DEFINE_PER_CPU(unsigned long, cpu_dr7);
EXPORT_PER_CPU_SYMBOL(cpu_dr7);
+/* Sequence number of the per-CPU DR7 state. */
+DEFINE_PER_CPU(unsigned int, cpu_dr7_seq);
/* Per cpu debug address registers values */
static DEFINE_PER_CPU(unsigned long, cpu_debugreg[HBP_NUM]);
@@ -97,38 +99,95 @@ int decode_dr7(unsigned long dr7, int bpnum, unsigned *len, unsigned *type)
int arch_install_hw_breakpoint(struct perf_event *bp)
{
struct arch_hw_breakpoint *info = counter_arch_bp(bp);
- unsigned long *dr7;
+ unsigned int seq;
int i;
lockdep_assert_irqs_disabled();
- for (i = 0; i < HBP_NUM; i++) {
- struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
+ if (perf_guest_in_guest())
+ return -EBUSY;
- if (!*slot) {
- *slot = bp;
+ for (i = 0; i < HBP_NUM; i++) {
+ if (!this_cpu_cmpxchg(bp_per_reg[i], NULL, bp))
break;
- }
}
if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
return -EBUSY;
- set_debugreg(info->address, i);
- __this_cpu_write(cpu_debugreg[i], info->address);
+ do {
+ seq = this_cpu_inc_return(cpu_dr7_seq);
+ this_cpu_write(cpu_debugreg[i], info->address);
+ barrier();
+ set_debugreg(info->address, i);
+ if (info->mask)
+ amd_set_dr_addr_mask(info->mask, i);
+ this_cpu_or(cpu_dr7, encode_dr7(i, info->len, info->type));
+ barrier();
+ set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);
+ barrier();
+ } while (seq != this_cpu_read(cpu_dr7_seq));
- dr7 = this_cpu_ptr(&cpu_dr7);
- *dr7 |= encode_dr7(i, info->len, info->type);
+ return 0;
+}
- /*
- * Ensure we first write cpu_dr7 before we set the DR7 register.
- * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
- */
- barrier();
+/*
+ * Modify the address of an installed perf counter breakpoint on local CPU.
+ *
+ * This updates the debug address register and per-CPU shadow state without
+ * changing slot allocation or DR7. The core layer (modify_local_hw_breakpoint_addr())
+ * manages updating bp->attr.bp_addr and counter_arch_bp(bp)->address.
+ *
+ * Atomic: called with IRQs disabled and handles variables and registers
+ * local to this CPU.
+ */
+int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+ struct arch_hw_breakpoint hw;
+ struct perf_event_attr attr = bp->attr;
+ unsigned int seq;
+ int i, ret;
+
+ lockdep_assert_irqs_disabled();
- set_debugreg(*dr7, 7);
- if (info->mask)
- amd_set_dr_addr_mask(info->mask, i);
+ if (perf_guest_in_guest())
+ return -EBUSY;
+
+ attr.bp_addr = addr;
+ ret = hw_breakpoint_arch_parse(bp, &attr, &hw);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < HBP_NUM; i++) {
+ if (this_cpu_read(bp_per_reg[i]) == bp)
+ break;
+ }
+
+ if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
+ return -ENOENT;
+
+ do {
+ seq = this_cpu_inc_return(cpu_dr7_seq);
+
+ /* Disable breakpoint in shadow DR7 and hardware DR7 */
+ this_cpu_and(cpu_dr7, ~__encode_dr7(i, info->len, info->type));
+ barrier();
+ set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);
+ barrier();
+
+ this_cpu_write(cpu_debugreg[i], addr);
+ barrier();
+ set_debugreg(addr, i);
+ barrier();
+
+ /* Re-enable breakpoint in shadow DR7 and hardware DR7 */
+ this_cpu_or(cpu_dr7, encode_dr7(i, info->len, info->type));
+ barrier();
+ set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);
+ barrier();
+ } while (seq != this_cpu_read(cpu_dr7_seq));
return 0;
}
@@ -146,36 +205,34 @@ void arch_uninstall_hw_breakpoint(struct perf_event *bp)
{
struct arch_hw_breakpoint *info = counter_arch_bp(bp);
unsigned long dr7;
+ unsigned int seq;
int i;
lockdep_assert_irqs_disabled();
for (i = 0; i < HBP_NUM; i++) {
- struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
-
- if (*slot == bp) {
- *slot = NULL;
+ if (this_cpu_read(bp_per_reg[i]) == bp)
break;
- }
}
if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
return;
- dr7 = this_cpu_read(cpu_dr7);
- dr7 &= ~__encode_dr7(i, info->len, info->type);
-
- set_debugreg(dr7, 7);
- if (info->mask)
- amd_set_dr_addr_mask(0, i);
-
- /*
- * Ensure the write to cpu_dr7 is after we've set the DR7 register.
- * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
- */
- barrier();
-
- this_cpu_write(cpu_dr7, dr7);
+ do {
+ seq = this_cpu_inc_return(cpu_dr7_seq);
+ dr7 = this_cpu_read(cpu_dr7);
+ dr7 &= ~__encode_dr7(i, info->len, info->type);
+ set_debugreg(dr7 | DR7_FIXED_1, 7);
+ if (info->mask)
+ amd_set_dr_addr_mask(0, i);
+ barrier();
+ this_cpu_and(cpu_dr7,
+ ~__encode_dr7(i, info->len, info->type));
+ barrier();
+ } while (seq != this_cpu_read(cpu_dr7_seq));
+
+ WARN_ONCE(this_cpu_cmpxchg(bp_per_reg[i], bp, NULL) != bp,
+ "Can't release breakpoint slot");
}
static int arch_bp_generic_len(int x86_len)
@@ -309,13 +366,14 @@ static inline bool within_cpu_entry(unsigned long addr, unsigned long end)
sizeof(struct tlb_state)))
return true;
- /*
- * When in guest (X86_FEATURE_HYPERVISOR), local_db_save()
- * will read per-cpu cpu_dr7 before clear dr7 register.
- */
+ /* local_db_save() reads this state before clearing DR7. */
if (within_area(addr, end, (unsigned long)&per_cpu(cpu_dr7, cpu),
sizeof(cpu_dr7)))
return true;
+ if (within_area(addr, end,
+ (unsigned long)&per_cpu(cpu_dr7_seq, cpu),
+ sizeof(cpu_dr7_seq)))
+ return true;
}
return false;
@@ -483,12 +541,18 @@ void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
void hw_breakpoint_restore(void)
{
- set_debugreg(__this_cpu_read(cpu_debugreg[0]), 0);
- set_debugreg(__this_cpu_read(cpu_debugreg[1]), 1);
- set_debugreg(__this_cpu_read(cpu_debugreg[2]), 2);
- set_debugreg(__this_cpu_read(cpu_debugreg[3]), 3);
- set_debugreg(DR6_RESERVED, 6);
- set_debugreg(__this_cpu_read(cpu_dr7), 7);
+ unsigned int seq;
+
+ do {
+ seq = this_cpu_inc_return(cpu_dr7_seq);
+ set_debugreg(this_cpu_read(cpu_debugreg[0]), 0);
+ set_debugreg(this_cpu_read(cpu_debugreg[1]), 1);
+ set_debugreg(this_cpu_read(cpu_debugreg[2]), 2);
+ set_debugreg(this_cpu_read(cpu_debugreg[3]), 3);
+ set_debugreg(DR6_RESERVED, 6);
+ set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);
+ barrier();
+ } while (seq != this_cpu_read(cpu_dr7_seq));
}
EXPORT_SYMBOL_FOR_KVM(hw_breakpoint_restore);
diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c
index 3c9f60d6ca5a7..c3806f7bd3e83 100644
--- a/arch/x86/kernel/nmi.c
+++ b/arch/x86/kernel/nmi.c
@@ -531,11 +531,12 @@ enum nmi_states {
};
static DEFINE_PER_CPU(enum nmi_states, nmi_state);
static DEFINE_PER_CPU(unsigned long, nmi_cr2);
-static DEFINE_PER_CPU(unsigned long, nmi_dr7);
DEFINE_IDTENTRY_RAW(exc_nmi)
{
irqentry_state_t irq_state;
+ unsigned long dr7;
+ unsigned int dr7_seq;
struct nmi_stats *nsp = this_cpu_ptr(&nmi_stats);
/*
@@ -572,7 +573,7 @@ DEFINE_IDTENTRY_RAW(exc_nmi)
*/
sev_es_ist_enter(regs);
- this_cpu_write(nmi_dr7, local_db_save());
+ local_db_save(&dr7, &dr7_seq);
irq_state = irqentry_nmi_enter(regs);
@@ -594,7 +595,7 @@ DEFINE_IDTENTRY_RAW(exc_nmi)
irqentry_nmi_exit(regs, irq_state);
- local_db_restore(this_cpu_read(nmi_dr7));
+ local_db_restore(dr7, dr7_seq);
sev_es_ist_exit();
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 30aa8369957e9..018abe736285d 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -1231,8 +1231,12 @@ static noinstr void exc_debug_kernel(struct pt_regs *regs, unsigned long dr6)
* it results in an endless recursion and stack overflow. Thus we stay
* with the IDT approach, i.e., save DR7 and disable #DB.
*/
- unsigned long dr7 = local_db_save();
- irqentry_state_t irq_state = irqentry_nmi_enter(regs);
+ unsigned long dr7;
+ unsigned int dr7_seq;
+ irqentry_state_t irq_state;
+
+ local_db_save(&dr7, &dr7_seq);
+ irq_state = irqentry_nmi_enter(regs);
instrumentation_begin();
/*
@@ -1289,7 +1293,7 @@ static noinstr void exc_debug_kernel(struct pt_regs *regs, unsigned long dr6)
instrumentation_end();
irqentry_nmi_exit(regs, irq_state);
- local_db_restore(dr7);
+ local_db_restore(dr7, dr7_seq);
}
static noinstr void exc_debug_user(struct pt_regs *regs, unsigned long dr6)
diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
index db199d653dd1a..bf65c7fffd996 100644
--- a/include/linux/hw_breakpoint.h
+++ b/include/linux/hw_breakpoint.h
@@ -81,6 +81,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
void *context);
+int modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr);
+
extern int register_perf_hw_breakpoint(struct perf_event *bp);
extern void unregister_hw_breakpoint(struct perf_event *bp);
extern void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events);
@@ -124,6 +127,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
void *context) { return NULL; }
static inline int
+modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr) { return -EOPNOTSUPP; }
+static inline int
register_perf_hw_breakpoint(struct perf_event *bp) { return -ENOSYS; }
static inline void unregister_hw_breakpoint(struct perf_event *bp) { }
static inline void
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 5842552294c19..194e69270460e 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1677,8 +1677,8 @@ struct perf_guest_info_callbacks {
unsigned int (*state)(void);
unsigned long (*get_ip)(void);
unsigned int (*handle_intel_pt_intr)(void);
-
void (*handle_mediated_pmi)(void);
+ bool (*in_guest)(void);
};
#ifdef CONFIG_GUEST_PERF_EVENTS
@@ -1689,6 +1689,7 @@ DECLARE_STATIC_CALL(__perf_guest_state, *perf_guest_cbs->state);
DECLARE_STATIC_CALL(__perf_guest_get_ip, *perf_guest_cbs->get_ip);
DECLARE_STATIC_CALL(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->handle_intel_pt_intr);
DECLARE_STATIC_CALL(__perf_guest_handle_mediated_pmi, *perf_guest_cbs->handle_mediated_pmi);
+DECLARE_STATIC_CALL(__perf_guest_in_guest, *perf_guest_cbs->in_guest);
static inline unsigned int perf_guest_state(void)
{
@@ -1710,6 +1711,11 @@ static inline void perf_guest_handle_mediated_pmi(void)
static_call(__perf_guest_handle_mediated_pmi)();
}
+static inline bool perf_guest_in_guest(void)
+{
+ return static_call(__perf_guest_in_guest)();
+}
+
extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
@@ -1718,6 +1724,7 @@ extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callback
static inline unsigned int perf_guest_state(void) { return 0; }
static inline unsigned long perf_guest_get_ip(void) { return 0; }
static inline unsigned int perf_guest_handle_intel_pt_intr(void) { return 0; }
+static inline bool perf_guest_in_guest(void) { return false; }
#endif /* !CONFIG_GUEST_PERF_EVENTS */
diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 5cbd09c8be8d5..f81a5308c1167 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -337,6 +337,7 @@ enum {
TRACE_EVENT_FL_UPROBE_BIT,
TRACE_EVENT_FL_EPROBE_BIT,
TRACE_EVENT_FL_FPROBE_BIT,
+ TRACE_EVENT_FL_WPROBE_BIT,
TRACE_EVENT_FL_CUSTOM_BIT,
TRACE_EVENT_FL_TEST_STR_BIT,
};
@@ -367,6 +368,7 @@ enum {
TRACE_EVENT_FL_UPROBE = (1 << TRACE_EVENT_FL_UPROBE_BIT),
TRACE_EVENT_FL_EPROBE = (1 << TRACE_EVENT_FL_EPROBE_BIT),
TRACE_EVENT_FL_FPROBE = (1 << TRACE_EVENT_FL_FPROBE_BIT),
+ TRACE_EVENT_FL_WPROBE = (1 << TRACE_EVENT_FL_WPROBE_BIT),
TRACE_EVENT_FL_CUSTOM = (1 << TRACE_EVENT_FL_CUSTOM_BIT),
TRACE_EVENT_FL_TEST_STR = (1 << TRACE_EVENT_FL_TEST_STR_BIT),
};
@@ -736,6 +738,7 @@ enum event_trigger_type {
ETT_EVENT_HIST = (1 << 4),
ETT_HIST_ENABLE = (1 << 5),
ETT_EVENT_EPROBE = (1 << 6),
+ ETT_EVENT_WPROBE = (1 << 7),
};
extern int filter_match_preds(struct event_filter *filter, void *rec);
diff --git a/kernel/events/core.c b/kernel/events/core.c
index a6c8e38a31104..c5dead7c5f71d 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -7749,6 +7749,7 @@ DEFINE_STATIC_CALL_RET0(__perf_guest_state, *perf_guest_cbs->state);
DEFINE_STATIC_CALL_RET0(__perf_guest_get_ip, *perf_guest_cbs->get_ip);
DEFINE_STATIC_CALL_RET0(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->handle_intel_pt_intr);
DEFINE_STATIC_CALL_RET0(__perf_guest_handle_mediated_pmi, *perf_guest_cbs->handle_mediated_pmi);
+DEFINE_STATIC_CALL_RET0(__perf_guest_in_guest, *perf_guest_cbs->in_guest);
void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
{
@@ -7767,6 +7768,9 @@ void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
if (cbs->handle_mediated_pmi)
static_call_update(__perf_guest_handle_mediated_pmi,
cbs->handle_mediated_pmi);
+
+ if (cbs->in_guest)
+ static_call_update(__perf_guest_in_guest, cbs->in_guest);
}
EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks);
@@ -7780,6 +7784,7 @@ void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
static_call_update(__perf_guest_get_ip, (void *)&__static_call_return0);
static_call_update(__perf_guest_handle_intel_pt_intr, (void *)&__static_call_return0);
static_call_update(__perf_guest_handle_mediated_pmi, (void *)&__static_call_return0);
+ static_call_update(__perf_guest_in_guest, (void *)&__static_call_return0);
synchronize_rcu();
}
EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
diff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c
index 789add0c185a0..69bfb17b87c41 100644
--- a/kernel/events/hw_breakpoint.c
+++ b/kernel/events/hw_breakpoint.c
@@ -888,6 +888,54 @@ void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events)
}
EXPORT_SYMBOL_GPL(unregister_wide_hw_breakpoint);
+/**
+ * modify_local_hw_breakpoint_addr - update a local breakpoint address
+ * @bp: the hwbp perf event for this CPU
+ * @addr: the new address for @bp
+ *
+ * Update only the address of an installed breakpoint on the local CPU without
+ * releasing and reserving its hardware slot. The caller must update other CPUs.
+ * Return 0, or -EOPNOTSUPP if the architecture does not support this operation.
+ *
+ * Note: bp->attr.bp_addr and counter_arch_bp(bp)->address are updated before
+ * modifying hardware registers so that interrupt/NMI handlers and CPU entry
+ * paths observe the new address during the transition. If an NMI hits the old
+ * breakpoint address before the hardware register update finishes, the event
+ * handler will see bp->attr.bp_addr already pointing to @addr. If the
+ * architecture update fails, both are rolled back to the previous address.
+ * Callers must be prepared for this behavior.
+ */
+#ifdef CONFIG_HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
+int modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr)
+{
+ unsigned long old_addr;
+ int ret;
+
+ lockdep_assert_irqs_disabled();
+
+ old_addr = bp->attr.bp_addr;
+ WRITE_ONCE(counter_arch_bp(bp)->address, addr);
+ WRITE_ONCE(bp->attr.bp_addr, addr);
+
+ ret = arch_modify_local_hw_breakpoint_addr(bp, addr);
+ if (ret) {
+ WRITE_ONCE(counter_arch_bp(bp)->address, old_addr);
+ WRITE_ONCE(bp->attr.bp_addr, old_addr);
+ return ret;
+ }
+
+ return 0;
+}
+#else
+int modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+EXPORT_SYMBOL_GPL(modify_local_hw_breakpoint_addr);
+
/**
* hw_breakpoint_is_used - check if breakpoints are currently used
*
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 0ab5916575a97..5fd8ed63c516d 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -862,6 +862,30 @@ config EPROBE_EVENTS
convert the type of an event field. For example, turn an
address into a string.
+config WPROBE_EVENTS
+ bool "Enable wprobe-based dynamic events"
+ depends on TRACING
+ depends on HAVE_HW_BREAKPOINT
+ depends on HAVE_POST_BREAKPOINT_HOOK
+ select PROBE_EVENTS
+ select DYNAMIC_EVENTS
+ help
+ This allows the user to add watchpoint tracing events based on
+ hardware breakpoints on the fly via the ftrace interface.
+
+ Those events can be inserted wherever hardware breakpoints can be
+ set, and record accessed memory address and values.
+
+config WPROBE_TRIGGERS
+ depends on WPROBE_EVENTS
+ depends on HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
+ bool
+ default y
+ help
+ This adds an event trigger which will set the wprobe on a specific
+ field of an event. This allows user to trace the memory access of
+ an address pointed by the event field.
+
config BPF_EVENTS
depends on BPF_SYSCALL
depends on (KPROBE_EVENTS || UPROBE_EVENTS) && PERF_EVENTS
diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile
index f934ff586bd43..141c8323de20b 100644
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@ -126,6 +126,7 @@ obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
obj-$(CONFIG_FPROBE) += fprobe.o
obj-$(CONFIG_RETHOOK) += rethook.o
obj-$(CONFIG_FPROBE_EVENTS) += trace_fprobe.o
+obj-$(CONFIG_WPROBE_EVENTS) += trace_wprobe.o
obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
obj-$(CONFIG_RV) += rv/
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index a946e0183fd1e..61d7d8bfb1577 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4295,7 +4295,7 @@ static const char readme_msg[] =
"\t\t\t Write into this file to define/undefine new trace events.\n"
#endif
#if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS) || \
- defined(CONFIG_FPROBE_EVENTS)
+ defined(CONFIG_FPROBE_EVENTS) || defined(CONFIG_WPROBE_EVENTS)
"\t accepts: event-definitions (one definition per line)\n"
#if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS)
"\t Format: p[:[<group>/][<event>]] <place> [<args>]\n"
@@ -4305,6 +4305,9 @@ static const char readme_msg[] =
"\t f[:[<group>/][<event>]] <func-name>[%return] [<args>]\n"
"\t t[:[<group>/][<event>]] <tracepoint> [<args>]\n"
#endif
+#ifdef CONFIG_WPROBE_EVENTS
+ "\t w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>] [<args>]\n"
+#endif
#ifdef CONFIG_HIST_TRIGGERS
"\t s:[synthetic/]<event> <field> [<field>]\n"
#endif
@@ -4359,6 +4362,10 @@ static const char readme_msg[] =
"\t trigger: traceon, traceoff\n"
"\t enable_event:<system>:<event>\n"
"\t disable_event:<system>:<event>\n"
+#ifdef CONFIG_WPROBE_TRIGGERS
+ "\t set_wprobe:<wprobe-event>:<field>[+|-<offset>]\n"
+ "\t clear_wprobe:<wprobe-event>[:<field>[+|-<offset>]]\n"
+#endif
#ifdef CONFIG_HIST_TRIGGERS
"\t enable_hist:<system>:<event>\n"
"\t disable_hist:<system>:<event>\n"
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 74a7a50d1e78f..a61f840b18bdb 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -179,6 +179,11 @@ struct fexit_trace_entry_head {
unsigned long ret_ip;
};
+struct wprobe_trace_entry_head {
+ struct trace_entry ent;
+ unsigned long ip;
+};
+
#define TRACE_BUF_SIZE 1024
struct trace_array;
@@ -1978,6 +1983,7 @@ trigger_data_alloc(struct event_command *cmd_ops, char *cmd, char *param,
void *private_data);
extern void trigger_data_free(struct event_trigger_data *data);
extern int event_trigger_init(struct event_trigger_data *data);
+extern void event_trigger_free(struct event_trigger_data *data);
extern int trace_event_trigger_enable_disable(struct trace_event_file *file,
int trigger_enable);
extern void update_cond_flag(struct trace_event_file *file);
diff --git a/kernel/trace/trace_events_trigger.c b/kernel/trace/trace_events_trigger.c
index 149300cc5e8ad..120e6ec634b76 100644
--- a/kernel/trace/trace_events_trigger.c
+++ b/kernel/trace/trace_events_trigger.c
@@ -589,7 +589,7 @@ int event_trigger_init(struct event_trigger_data *data)
* Usually used directly as the @free method in event trigger
* implementations.
*/
-static void
+void
event_trigger_free(struct event_trigger_data *data)
{
if (WARN_ON_ONCE(data->ref <= 0))
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index 804442b2f7d2b..ccf489635ad7f 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -20,7 +20,7 @@
#undef C
#define C(a, b) b
-static const char *trace_probe_err_text[] = { ERRORS };
+const char *trace_probe_err_text[] = { ERRORS };
static const char *reserved_field_names[] = {
"common_type",
@@ -890,6 +890,16 @@ static int query_btf_struct(const char *sname, struct traceprobe_parse_context *
ctx->struct_btf = NULL;
}
+ if (ctx->btf) {
+ id = btf_find_by_name_kind(ctx->btf, sname, BTF_KIND_STRUCT);
+ if (id > 0) {
+ btf_get(ctx->btf);
+ ctx->struct_btf = ctx->btf;
+ ctx->last_struct = btf_type_by_id(ctx->struct_btf, id);
+ return 0;
+ }
+ }
+
id = bpf_find_btf_id(sname, BTF_KIND_STRUCT, &btf);
if (id < 0)
return id;
@@ -965,7 +975,8 @@ static int handle_typecast(char *arg, struct traceprobe_parse_context *ctx)
if (!(tparg_is_event_probe(ctx->flags) ||
tparg_is_function_entry(ctx->flags) ||
- tparg_is_function_return(ctx->flags))) {
+ tparg_is_function_return(ctx->flags) ||
+ tparg_is_wprobe(ctx->flags))) {
trace_probe_log_err(ctx->offset, NOSUP_BTFARG);
return -EOPNOTSUPP;
}
@@ -1403,6 +1414,23 @@ static int parse_probe_vars(char *orig_arg, const struct fetch_type *t,
return 0;
}
+ /* wprobe only support "$addr" and "$value" variable */
+ if (ctx->flags & TPARG_FL_WPROBE) {
+ if (!strcmp(arg, "addr")) {
+ code->op = FETCH_OP_BADDR;
+ return 0;
+ }
+ if (!strcmp(arg, "value")) {
+ code->op = FETCH_OP_BADDR;
+ code++;
+ code->op = FETCH_OP_DEREF;
+ code->offset = 0;
+ *pcode = code;
+ return 0;
+ }
+ goto inval;
+ }
+
if (strcmp(arg, "comm") == 0 || strcmp(arg, "COMM") == 0) {
code->op = FETCH_OP_COMM;
return 0;
@@ -1462,8 +1490,8 @@ static int parse_probe_arg_register(char *arg, struct fetch_insn *code,
{
int ret;
- if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE)) {
- /* eprobe and fprobe do not handle registers */
+ if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE | TPARG_FL_WPROBE)) {
+ /* eprobe, fprobe and wprobe do not handle registers */
trace_probe_log_err(ctx->offset, BAD_VAR);
return -EINVAL;
}
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index d1fb3520700fb..c54c554b1949d 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -90,6 +90,7 @@ typedef int (*print_type_func_t)(struct trace_seq *, void *, void *);
FETCH_OP(STACK, param), /* Stack: .param = index */ \
FETCH_OP(STACKP, none), /* Stack pointer */ \
FETCH_OP(RETVAL, none), /* Return value */ \
+ FETCH_OP(BADDR, none), /* Break address */ \
FETCH_OP(IMM, imm), /* Immediate: .immediate */ \
FETCH_OP(COMM, none), /* Current comm */ \
FETCH_OP(CURRENT, none), /* Current task_struct address */\
@@ -420,6 +421,7 @@ static inline int traceprobe_get_entry_data_size(struct trace_probe *tp)
#define TPARG_FL_USER BIT(4)
#define TPARG_FL_FPROBE BIT(5)
#define TPARG_FL_TPOINT BIT(6)
+#define TPARG_FL_WPROBE BIT(7)
#define TPARG_FL_LOC_MASK GENMASK(4, 0)
static inline bool tparg_is_function_entry(unsigned int flags)
@@ -437,6 +439,11 @@ static inline bool tparg_is_event_probe(unsigned int flags)
return !!(flags & TPARG_FL_TEVENT);
}
+static inline bool tparg_is_wprobe(unsigned int flags)
+{
+ return !!(flags & TPARG_FL_WPROBE);
+}
+
/* Each typecast consumes nested level. So the max number of typecast is 8. */
#define TRACEPROBE_MAX_NESTED_LEVEL 8
@@ -546,6 +553,10 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(ARG_TOO_LONG, "Argument expression is too long"), \
C(ARRAY_NO_CLOSE, "Array is not closed"), \
C(ARRAY_TOO_BIG, "Array number is too big"), \
+ C(BAD_ACCESS_ADDR, "Invalid access memory address"), \
+ C(BAD_ACCESS_FMT, "Access memory address requires @"), \
+ C(BAD_ACCESS_LEN, "This memory access length is not supported"), \
+ C(BAD_ACCESS_TYPE, "Bad memory access type"), \
C(BAD_ADDR_SUFFIX, "Invalid probed address suffix"), \
C(BAD_ARG_NAME, "Argument name must follow the same rules as C identifiers"), \
C(BAD_ARG_NUM, "Invalid argument number"), \
@@ -628,7 +639,14 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(TYPECAST_NOT_EVENT, "Typecasts are only for eprobe fields"), \
C(TYPECAST_REQ_FIELD, "Typecast requires a field access"), \
C(TYPECAST_SYM_OFFSET, "@SYM+/-OFFSET with typecast needs parentheses"), \
- C(USED_ARG_NAME, "This argument name is already used"),
+ C(USED_ARG_NAME, "This argument name is already used"), \
+ C(WPROBE_NO_MAXACT, "Watchpoint probe does not support maxactive"), \
+ C(WPROBE_NO_SIBLING, "Watchpoint probe does not support sibling probes"), \
+ C(WPROBE_ON_KPROBE, "Wprobe trigger is not supported on kprobe event"), \
+ C(WPROBE_NOT_FOUND, "Target wprobe event is not found"), \
+ C(WPROBE_BUSY, "Target wprobe event is already enabled"), \
+ C(WPROBE_NEED_FIELD, "Wprobe trigger requires a target field"), \
+ C(WPROBE_BAD_FIELD, "Target field must be pointer size"),
#undef C
#define C(a, b) TP_ERR_##a
@@ -652,6 +670,8 @@ void __trace_probe_log_err(int offset, int err);
DEFINE_FREE(trace_probe_log_clear, const char *, if (_T) trace_probe_log_clear())
+extern const char *trace_probe_err_text[];
+
#define trace_probe_log_err(offs, err) \
__trace_probe_log_err(offs, TP_ERR_##err)
diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c
new file mode 100644
index 0000000000000..121310b72522f
--- /dev/null
+++ b/kernel/trace/trace_wprobe.c
@@ -0,0 +1,1668 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hardware-breakpoint-based tracing events
+ *
+ * Copyright (C) 2023, Masami Hiramatsu <mhiramat@kernel.org>
+ */
+#define pr_fmt(fmt) "trace_wprobe: " fmt
+
+#include <linux/atomic.h>
+#include <linux/compiler.h>
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/errno.h>
+#include <linux/hw_breakpoint.h>
+#include <linux/kallsyms.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/perf_event.h>
+#include <linux/rculist.h>
+#include <linux/security.h>
+#include <linux/spinlock.h>
+#include <linux/tracepoint.h>
+#include <linux/uaccess.h>
+#include <linux/workqueue.h>
+#include <linux/irq_work.h>
+#include <linux/preempt.h>
+
+#include <asm/ptrace.h>
+
+#include "trace.h"
+#include "trace_btf.h"
+#include "trace_dynevent.h"
+#include "trace_probe.h"
+#include "trace_probe_kernel.h"
+#include "trace_probe_tmpl.h"
+#include "trace_output.h"
+
+#define WPROBE_EVENT_SYSTEM "wprobes"
+
+static int trace_wprobe_create(const char *raw_command);
+static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev);
+static int trace_wprobe_release(struct dyn_event *ev);
+static bool trace_wprobe_is_busy(struct dyn_event *ev);
+static bool trace_wprobe_match(const char *system, const char *event,
+ int argc, const char **argv, struct dyn_event *ev);
+
+static struct dyn_event_operations trace_wprobe_ops = {
+ .create = trace_wprobe_create,
+ .show = trace_wprobe_show,
+ .is_busy = trace_wprobe_is_busy,
+ .free = trace_wprobe_release,
+ .match = trace_wprobe_match,
+};
+
+struct trace_wprobe {
+ struct dyn_event devent;
+ struct list_head active_list;
+ struct perf_event * __percpu *bp_event;
+ unsigned long addr;
+ int offset;
+ int len;
+ int type;
+ const char *symbol;
+ raw_spinlock_t lock;
+ struct irq_work irq_work;
+ struct work_struct work;
+ atomic_t missed;
+ /*
+ * work_pending is set to 1 before irq_work_queue() and cleared to 0
+ * after wprobe_work_func() finishes on_each_cpu(). This prevents a
+ * new trigger from overwriting tw->addr while the work is propagating
+ * the old address to per-CPU debug registers via IPI.
+ */
+ atomic_t work_pending;
+ struct trace_probe tp;
+};
+
+static bool is_trace_wprobe(struct dyn_event *ev)
+{
+ return ev->ops == &trace_wprobe_ops;
+}
+
+static struct trace_wprobe *to_trace_wprobe(struct dyn_event *ev)
+{
+ return container_of(ev, struct trace_wprobe, devent);
+}
+
+static DEFINE_MUTEX(wprobe_mutex);
+static LIST_HEAD(active_wprobes);
+
+#define for_each_trace_wprobe(pos, dpos) \
+ for_each_dyn_event(dpos) \
+ if (is_trace_wprobe(dpos) && (pos = to_trace_wprobe(dpos)))
+
+static bool trace_wprobe_is_busy(struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+
+ return trace_probe_is_enabled(&tw->tp);
+}
+
+static bool trace_wprobe_match(const char *system, const char *event,
+ int argc, const char **argv, struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+
+ if (event[0] != '\0' && strcmp(trace_probe_name(&tw->tp), event))
+ return false;
+
+ if (system && strcmp(trace_probe_group_name(&tw->tp), system))
+ return false;
+
+ return trace_probe_match_command_args(&tw->tp, argc, argv);
+}
+
+/*
+ * Note that we don't verify the fetch_insn code, since it does not come
+ * from user space.
+ */
+static int
+process_fetch_insn(struct fetch_insn *code, void *rec, void *edata,
+ void *dest, void *base)
+{
+ void *baddr = rec;
+ unsigned long val;
+ int ret;
+
+retry:
+ /* 1st stage: get value from context */
+ switch (code->op) {
+ case FETCH_OP_BADDR:
+ val = (unsigned long)baddr;
+ break;
+ case FETCH_NOP_SYMBOL: /* Ignore a place holder */
+ code++;
+ goto retry;
+ default:
+ ret = process_common_fetch_insn(code, &val);
+ if (ret < 0)
+ return ret;
+ }
+ code++;
+
+ return process_fetch_insn_bottom(code, val, dest, base);
+}
+NOKPROBE_SYMBOL(process_fetch_insn)
+
+static void wprobe_trace_handler(struct trace_wprobe *tw,
+ unsigned long addr,
+ struct pt_regs *regs,
+ struct trace_event_file *trace_file)
+{
+ struct wprobe_trace_entry_head *entry;
+ struct trace_event_call *call = trace_probe_event_call(&tw->tp);
+ struct trace_event_buffer fbuffer;
+ int dsize;
+
+ if (WARN_ON_ONCE(call != trace_file->event_call))
+ return;
+
+ if (trace_trigger_soft_disabled(trace_file))
+ return;
+
+ if (READ_ONCE(tw->addr) != addr)
+ return;
+
+ dsize = __get_data_size(&tw->tp, (void *)addr, NULL);
+
+ entry = trace_event_buffer_reserve(&fbuffer, trace_file,
+ sizeof(*entry) + tw->tp.size + dsize);
+ if (!entry)
+ return;
+
+ entry->ip = instruction_pointer(regs);
+ store_trace_args(&entry[1], &tw->tp, (void *)addr, NULL, sizeof(*entry), dsize);
+
+ fbuffer.regs = regs;
+ trace_event_buffer_commit(&fbuffer);
+}
+
+static void wprobe_perf_handler(struct perf_event *bp,
+ struct perf_sample_data *data,
+ struct pt_regs *regs)
+{
+ struct trace_wprobe *tw = bp->overflow_handler_context;
+ struct event_file_link *link;
+ unsigned long addr = bp->attr.bp_addr;
+
+ trace_probe_for_each_link_rcu(link, &tw->tp)
+ wprobe_trace_handler(tw, addr, regs, link->file);
+}
+
+static int __register_trace_wprobe(struct trace_wprobe *tw)
+{
+ struct perf_event_attr attr;
+ int i, ret;
+
+ if (tw->bp_event)
+ return -EINVAL;
+
+ for (i = 0; i < tw->tp.nr_args; i++) {
+ ret = traceprobe_update_arg(&tw->tp.args[i]);
+ if (ret)
+ return ret;
+ }
+
+ hw_breakpoint_init(&attr);
+ attr.bp_addr = tw->addr;
+ attr.bp_len = tw->len;
+ attr.bp_type = tw->type;
+
+ tw->bp_event = register_wide_hw_breakpoint(&attr, wprobe_perf_handler, tw);
+ if (IS_ERR_PCPU(tw->bp_event)) {
+ int ret = PTR_ERR_PCPU(tw->bp_event);
+
+ tw->bp_event = NULL;
+ return ret;
+ }
+
+ mutex_lock(&wprobe_mutex);
+ list_add(&tw->active_list, &active_wprobes);
+ mutex_unlock(&wprobe_mutex);
+
+ return 0;
+}
+
+static void __unregister_trace_wprobe(struct trace_wprobe *tw)
+{
+ if (tw->bp_event) {
+ mutex_lock(&wprobe_mutex);
+ list_del_init(&tw->active_list);
+ mutex_unlock(&wprobe_mutex);
+
+ irq_work_sync(&tw->irq_work);
+ cancel_work_sync(&tw->work);
+ unregister_wide_hw_breakpoint(tw->bp_event);
+ tw->bp_event = NULL;
+ }
+}
+
+static int wprobe_cpu_online(unsigned int cpu)
+{
+ struct trace_wprobe *tw;
+
+ mutex_lock(&wprobe_mutex);
+ list_for_each_entry(tw, &active_wprobes, active_list) {
+ if (tw->bp_event && !per_cpu(*tw->bp_event, cpu)) {
+ struct perf_event_attr attr;
+ struct perf_event *bp;
+
+ hw_breakpoint_init(&attr);
+ attr.bp_addr = READ_ONCE(tw->addr);
+ attr.bp_len = tw->len;
+ attr.bp_type = tw->type;
+
+ bp = perf_event_create_kernel_counter(&attr, cpu, NULL,
+ wprobe_perf_handler, tw);
+ if (!IS_ERR(bp))
+ per_cpu(*tw->bp_event, cpu) = bp;
+ }
+ }
+ mutex_unlock(&wprobe_mutex);
+
+ return 0;
+}
+
+static int wprobe_cpu_offline(unsigned int cpu)
+{
+ struct trace_wprobe *tw;
+
+ mutex_lock(&wprobe_mutex);
+ list_for_each_entry(tw, &active_wprobes, active_list) {
+ if (tw->bp_event) {
+ struct perf_event *bp = per_cpu(*tw->bp_event, cpu);
+
+ if (bp) {
+ per_cpu(*tw->bp_event, cpu) = NULL;
+ unregister_hw_breakpoint(bp);
+ }
+ }
+ }
+ mutex_unlock(&wprobe_mutex);
+
+ return 0;
+}
+
+static int trace_wprobe_update_local(struct trace_wprobe *tw, unsigned long addr)
+{
+ struct perf_event * __percpu *pevent;
+ struct perf_event *bp;
+
+ pevent = tw->bp_event;
+ if (!pevent)
+ return -EINVAL;
+
+ bp = *this_cpu_ptr(pevent);
+ if (!bp)
+ return -EINVAL;
+
+ if (bp->attr.bp_addr == addr)
+ return 0;
+
+ return modify_local_hw_breakpoint_addr(bp, addr);
+}
+
+static void wprobe_smp_update_func(void *info)
+{
+ struct trace_wprobe *tw = info;
+ unsigned long addr = READ_ONCE(tw->addr);
+
+ if (trace_wprobe_update_local(tw, addr))
+ atomic_inc(&tw->missed);
+}
+
+static void wprobe_work_func(struct work_struct *work)
+{
+ struct trace_wprobe *tw = container_of(work, struct trace_wprobe, work);
+
+ cpus_read_lock();
+ on_each_cpu(wprobe_smp_update_func, tw, true);
+ cpus_read_unlock();
+ /*
+ * Clear work_pending after all CPUs have updated their local debug
+ * registers. A new trigger may now update tw->addr and queue a new
+ * irq_work.
+ */
+ atomic_set(&tw->work_pending, 0);
+}
+
+static void wprobe_irq_work_func(struct irq_work *irq_work)
+{
+ struct trace_wprobe *tw = container_of(irq_work, struct trace_wprobe, irq_work);
+
+ schedule_work(&tw->work);
+}
+
+static void free_trace_wprobe(struct trace_wprobe *tw)
+{
+ if (tw) {
+ if (tw->work.func) {
+ irq_work_sync(&tw->irq_work);
+ cancel_work_sync(&tw->work);
+ }
+ trace_probe_cleanup(&tw->tp);
+ kfree(tw->symbol);
+ kfree(tw);
+ }
+}
+DEFINE_FREE(free_trace_wprobe, struct trace_wprobe *,
+ if (!IS_ERR_OR_NULL(_T))
+ free_trace_wprobe(_T))
+
+
+static struct trace_wprobe *alloc_trace_wprobe(const char *group,
+ const char *event,
+ const char *symbol,
+ int offset,
+ unsigned long addr,
+ int len, int type, int nargs)
+{
+ struct trace_wprobe *tw __free(free_trace_wprobe) = NULL;
+ int ret;
+
+ tw = kzalloc_flex(*tw, tp.args, nargs);
+ if (!tw)
+ return ERR_PTR(-ENOMEM);
+
+ raw_spin_lock_init(&tw->lock);
+ INIT_LIST_HEAD(&tw->active_list);
+ init_irq_work(&tw->irq_work, wprobe_irq_work_func);
+ INIT_WORK(&tw->work, wprobe_work_func);
+ atomic_set(&tw->missed, 0);
+ atomic_set(&tw->work_pending, 0);
+
+ if (symbol) {
+ tw->symbol = kstrdup(symbol, GFP_KERNEL);
+ if (!tw->symbol)
+ return ERR_PTR(-ENOMEM);
+ }
+ tw->offset = offset;
+ tw->addr = addr;
+ tw->len = len;
+ tw->type = type;
+
+ ret = trace_probe_init(&tw->tp, event, group, false, nargs);
+ if (ret < 0)
+ return ERR_PTR(ret);
+
+ dyn_event_init(&tw->devent, &trace_wprobe_ops);
+ return_ptr(tw);
+}
+
+static struct trace_wprobe *find_trace_wprobe(const char *event,
+ const char *group)
+{
+ struct dyn_event *pos;
+ struct trace_wprobe *tw;
+
+ for_each_trace_wprobe(tw, pos)
+ if (strcmp(trace_probe_name(&tw->tp), event) == 0 &&
+ strcmp(trace_probe_group_name(&tw->tp), group) == 0)
+ return tw;
+ return NULL;
+}
+
+static enum print_line_t
+print_wprobe_event(struct trace_iterator *iter, int flags,
+ struct trace_event *event)
+{
+ struct wprobe_trace_entry_head *field;
+ struct trace_seq *s = &iter->seq;
+ struct trace_probe *tp;
+
+ field = (struct wprobe_trace_entry_head *)iter->ent;
+ tp = trace_probe_primary_from_call(
+ container_of(event, struct trace_event_call, event));
+ if (WARN_ON_ONCE(!tp))
+ goto out;
+
+ trace_seq_printf(s, "%s: (", trace_probe_name(tp));
+
+ if (!seq_print_ip_sym_offset(s, field->ip, flags))
+ goto out;
+
+ trace_seq_putc(s, ')');
+
+ if (trace_probe_print_args(s, tp->args, tp->nr_args,
+ (u8 *)&field[1], field) < 0)
+ goto out;
+
+ trace_seq_putc(s, '\n');
+out:
+ return trace_handle_return(s);
+}
+
+static int wprobe_event_define_fields(struct trace_event_call *event_call)
+{
+ int ret;
+ struct wprobe_trace_entry_head field;
+ struct trace_probe *tp;
+
+ tp = trace_probe_primary_from_call(event_call);
+ if (WARN_ON_ONCE(!tp))
+ return -ENOENT;
+
+ DEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);
+
+ return traceprobe_define_arg_fields(event_call, sizeof(field), tp);
+}
+
+static struct trace_event_functions wprobe_funcs = {
+ .trace = print_wprobe_event
+};
+
+static struct trace_event_fields wprobe_fields_array[] = {
+ { .type = TRACE_FUNCTION_TYPE,
+ .define_fields = wprobe_event_define_fields },
+ {}
+};
+
+static int wprobe_register(struct trace_event_call *event,
+ enum trace_reg type, void *data);
+
+static inline void init_trace_event_call(struct trace_wprobe *tw)
+{
+ struct trace_event_call *call = trace_probe_event_call(&tw->tp);
+
+ call->event.funcs = &wprobe_funcs;
+ call->class->fields_array = wprobe_fields_array;
+ call->flags = TRACE_EVENT_FL_WPROBE;
+ call->class->reg = wprobe_register;
+}
+
+static int register_wprobe_event(struct trace_wprobe *tw)
+{
+ init_trace_event_call(tw);
+ return trace_probe_register_event_call(&tw->tp);
+}
+
+static int register_trace_wprobe_event(struct trace_wprobe *tw)
+{
+ struct trace_wprobe *old_tw;
+ int ret;
+
+ guard(mutex)(&event_mutex);
+
+ old_tw = find_trace_wprobe(trace_probe_name(&tw->tp),
+ trace_probe_group_name(&tw->tp));
+ if (old_tw) {
+ /*
+ * Wprobe does not support sibling probes because the event
+ * trigger (set_wprobe/clear_wprobe) identifies the target
+ * wprobe by its event name. Having multiple wprobes sharing
+ * the same event name would make the target ambiguous.
+ */
+ trace_probe_log_set_index(0);
+ trace_probe_log_err(0, WPROBE_NO_SIBLING);
+ return -EBUSY;
+ }
+
+ ret = register_wprobe_event(tw);
+ if (ret) {
+ trace_probe_log_set_index(0);
+ if (ret == -EEXIST)
+ trace_probe_log_err(0, EVENT_EXIST);
+ else if (ret != -ENOMEM)
+ trace_probe_log_err(0, FAIL_REG_PROBE);
+ return ret;
+ }
+
+ dyn_event_add(&tw->devent, trace_probe_event_call(&tw->tp));
+ return 0;
+}
+static int unregister_wprobe_event(struct trace_wprobe *tw)
+{
+ return trace_probe_unregister_event_call(&tw->tp);
+}
+
+static int unregister_trace_wprobe(struct trace_wprobe *tw)
+{
+ if (trace_probe_has_sibling(&tw->tp))
+ goto unreg;
+
+ if (trace_probe_is_enabled(&tw->tp))
+ return -EBUSY;
+
+ if (trace_event_dyn_busy(trace_probe_event_call(&tw->tp)))
+ return -EBUSY;
+
+ if (unregister_wprobe_event(tw))
+ return -EBUSY;
+
+unreg:
+ __unregister_trace_wprobe(tw);
+ dyn_event_remove(&tw->devent);
+ trace_probe_unlink(&tw->tp);
+
+ return 0;
+}
+
+static int enable_trace_wprobe(struct trace_event_call *call,
+ struct trace_event_file *file)
+{
+ struct trace_probe *tp;
+ struct trace_wprobe *tw;
+ bool enabled;
+ int ret = 0;
+
+ tp = trace_probe_primary_from_call(call);
+ if (WARN_ON_ONCE(!tp))
+ return -ENODEV;
+ enabled = trace_probe_is_enabled(tp);
+
+ if (file) {
+ ret = trace_probe_add_file(tp, file);
+ if (ret)
+ return ret;
+ } else {
+ trace_probe_set_flag(tp, TP_FLAG_PROFILE);
+ }
+
+ if (!enabled) {
+ list_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) {
+ ret = __register_trace_wprobe(tw);
+ if (ret < 0) {
+ struct trace_wprobe *tmp;
+
+ list_for_each_entry(tmp, trace_probe_probe_list(tp), tp.list) {
+ if (tmp == tw)
+ break;
+ __unregister_trace_wprobe(tmp);
+ }
+ if (file)
+ trace_probe_remove_file(tp, file);
+ else
+ trace_probe_clear_flag(tp, TP_FLAG_PROFILE);
+ return ret;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int disable_trace_wprobe(struct trace_event_call *call,
+ struct trace_event_file *file)
+{
+ struct trace_wprobe *tw;
+ struct trace_probe *tp;
+
+ tp = trace_probe_primary_from_call(call);
+ if (WARN_ON_ONCE(!tp))
+ return -ENODEV;
+
+ if (file) {
+ if (!trace_probe_get_file_link(tp, file))
+ return -ENOENT;
+ if (!trace_probe_has_single_file(tp))
+ goto out;
+ trace_probe_clear_flag(tp, TP_FLAG_TRACE);
+ } else {
+ trace_probe_clear_flag(tp, TP_FLAG_PROFILE);
+ }
+
+ if (!trace_probe_is_enabled(tp)) {
+ list_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) {
+ __unregister_trace_wprobe(tw);
+ }
+ }
+
+out:
+ if (file)
+ trace_probe_remove_file(tp, file);
+
+ return 0;
+}
+
+static int wprobe_register(struct trace_event_call *event,
+ enum trace_reg type, void *data)
+{
+ struct trace_event_file *file = data;
+
+ switch (type) {
+ case TRACE_REG_REGISTER:
+ return enable_trace_wprobe(event, file);
+ case TRACE_REG_UNREGISTER:
+ return disable_trace_wprobe(event, file);
+
+#ifdef CONFIG_PERF_EVENTS
+ case TRACE_REG_PERF_REGISTER:
+ case TRACE_REG_PERF_UNREGISTER:
+ case TRACE_REG_PERF_OPEN:
+ case TRACE_REG_PERF_CLOSE:
+ case TRACE_REG_PERF_ADD:
+ case TRACE_REG_PERF_DEL:
+ return -EOPNOTSUPP;
+#endif
+ }
+ return 0;
+}
+
+static bool trace_wprobe_is_valid_addr(unsigned long addr, int len)
+{
+ if (addr < TASK_SIZE)
+ return false;
+
+ if (addr & (len - 1))
+ return false;
+
+ return true;
+}
+
+#ifdef CONFIG_WPROBE_TRIGGERS
+static u64 wprobe_trigger_clear_target __aligned(8);
+#define WPROBE_DEFAULT_CLEAR_ADDRESS ((unsigned long)&wprobe_trigger_clear_target)
+#endif
+
+static int parse_address_spec(const char *spec, unsigned long *addr, int *type,
+ int *len, char **symbol, int *offset)
+{
+ char *_spec __free(kfree) = NULL;
+ int _len = HW_BREAKPOINT_LEN_4;
+ int _type = HW_BREAKPOINT_RW;
+ unsigned long _addr = 0;
+ int _offset = 0;
+ char *at, *col;
+
+ _spec = kstrdup(spec, GFP_KERNEL);
+ if (!_spec)
+ return -ENOMEM;
+
+ at = strchr(_spec, '@');
+ col = strchr(_spec, ':');
+
+ if (!at) {
+ trace_probe_log_err(0, BAD_ACCESS_FMT);
+ return -EINVAL;
+ }
+
+ if (at != _spec) {
+ *at = '\0';
+
+ if (strcmp(_spec, "r") == 0)
+ _type = HW_BREAKPOINT_R;
+ else if (strcmp(_spec, "w") == 0)
+ _type = HW_BREAKPOINT_W;
+ else if (strcmp(_spec, "rw") == 0)
+ _type = HW_BREAKPOINT_RW;
+ else {
+ trace_probe_log_err(0, BAD_ACCESS_TYPE);
+ return -EINVAL;
+ }
+ }
+
+ if (col) {
+ *col = '\0';
+ if (kstrtoint(col + 1, 0, &_len)) {
+ trace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN);
+ return -EINVAL;
+ }
+
+ switch (_len) {
+ case 1:
+ _len = HW_BREAKPOINT_LEN_1;
+ break;
+ case 2:
+ _len = HW_BREAKPOINT_LEN_2;
+ break;
+ case 4:
+ _len = HW_BREAKPOINT_LEN_4;
+ break;
+ case 8:
+ _len = HW_BREAKPOINT_LEN_8;
+ break;
+ default:
+ trace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN);
+ return -EINVAL;
+ }
+ }
+
+#ifdef CONFIG_WPROBE_TRIGGERS
+ if (strcmp(at + 1, "-1") == 0) {
+ _addr = WPROBE_DEFAULT_CLEAR_ADDRESS;
+ } else if (kstrtoul(at + 1, 0, &_addr) != 0) {
+#else
+ if (kstrtoul(at + 1, 0, &_addr) != 0) {
+#endif
+ char *off_str = strpbrk(at + 1, "+-");
+
+ if (off_str) {
+ if (kstrtoint(off_str, 0, &_offset) != 0) {
+ trace_probe_log_err(off_str - _spec, BAD_PROBE_ADDR);
+ return -EINVAL;
+ }
+ *off_str = '\0';
+ }
+ _addr = kallsyms_lookup_name(at + 1);
+ if (!_addr) {
+ trace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR);
+ return -ENOENT;
+ }
+ _addr += _offset;
+ *symbol = kstrdup(at + 1, GFP_KERNEL);
+ if (!*symbol)
+ return -ENOMEM;
+ }
+
+ if (!trace_wprobe_is_valid_addr(_addr, _len)) {
+ trace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR);
+ return -EINVAL;
+ }
+
+ *addr = _addr;
+ *offset = _offset;
+ *type = _type;
+ *len = _len;
+ return 0;
+}
+
+static int __trace_wprobe_create(int argc, const char *argv[])
+{
+ /*
+ * Argument syntax:
+ * w[:[GRP/][EVENT]] SPEC
+ *
+ * SPEC:
+ * [r|w|rw]@[ADDR|SYMBOL[+OFFS]][:LEN]
+ */
+ struct traceprobe_parse_context *ctx __free(traceprobe_parse_context) = NULL;
+ struct trace_wprobe *tw __free(free_trace_wprobe) = NULL;
+ const char *event = NULL, *group = WPROBE_EVENT_SYSTEM;
+ const char *tplog __free(trace_probe_log_clear) = NULL;
+ char *symbol __free(kfree) = NULL;
+ char *gbuf __free(kfree) = NULL;
+ char *ebuf __free(kfree) = NULL;
+ unsigned long addr;
+ int len, type, offset, i;
+ int ret;
+
+ if (argv[0][0] != 'w')
+ return -ECANCELED;
+
+ tplog = trace_probe_log_init("wprobe", argc, argv);
+
+ if (argc < 2) {
+ trace_probe_log_set_index(0);
+ trace_probe_log_err(0, NO_ARG_BODY);
+ return -EINVAL;
+ }
+
+ if (argv[0][1] != '\0') {
+ if (argv[0][1] != ':') {
+ trace_probe_log_set_index(0);
+ trace_probe_log_err(1, WPROBE_NO_MAXACT);
+ return -EINVAL;
+ }
+ event = &argv[0][2];
+ }
+
+ trace_probe_log_set_index(1);
+ ret = parse_address_spec(argv[1], &addr, &type, &len, &symbol, &offset);
+ if (ret < 0)
+ return ret;
+
+ trace_probe_log_set_index(0);
+ if (event) {
+ gbuf = kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL);
+ if (!gbuf)
+ return -ENOMEM;
+ ret = traceprobe_parse_event_name(&event, &group, gbuf,
+ event - argv[0]);
+ if (ret)
+ return ret;
+ }
+
+ if (!event) {
+ /* Make a new event name */
+ ebuf = kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL);
+ if (!ebuf)
+ return -ENOMEM;
+ if (symbol)
+ snprintf(ebuf, MAX_EVENT_NAME_LEN, "%s", symbol);
+ else
+ snprintf(ebuf, MAX_EVENT_NAME_LEN, "w_0x%lx", addr);
+ sanitize_event_name(ebuf);
+ event = ebuf;
+ }
+
+ argc -= 2; argv += 2;
+ if (argc > MAX_TRACE_ARGS) {
+ trace_probe_log_set_index(2);
+ trace_probe_log_err(0, TOO_MANY_ARGS);
+ return -E2BIG;
+ }
+ tw = alloc_trace_wprobe(group, event, symbol, offset, addr, len, type, argc);
+ if (IS_ERR(tw))
+ return PTR_ERR(tw);
+
+ ctx = kzalloc_obj(*ctx);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->flags = TPARG_FL_KERNEL | TPARG_FL_WPROBE;
+
+ /* parse arguments */
+ for (i = 0; i < argc; i++) {
+ trace_probe_log_set_index(i + 2);
+ ctx->offset = 0;
+ ret = traceprobe_parse_probe_arg(&tw->tp, i, argv[i], ctx);
+ if (ret)
+ return ret; /* This can be -ENOMEM */
+ }
+
+ ret = traceprobe_set_print_fmt(&tw->tp, PROBE_PRINT_NORMAL);
+ if (ret < 0)
+ return ret;
+
+ ret = register_trace_wprobe_event(tw);
+ if (!ret)
+ tw = NULL; /* To avoid free */
+
+ return ret;
+}
+
+static int trace_wprobe_create(const char *raw_command)
+{
+ return trace_probe_create(raw_command, __trace_wprobe_create);
+}
+
+static int trace_wprobe_release(struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+ int ret = unregister_trace_wprobe(tw);
+
+ if (!ret)
+ free_trace_wprobe(tw);
+ return ret;
+}
+
+static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+ const char *type_str;
+ int i, len;
+
+ seq_printf(m, "w:%s/%s", trace_probe_group_name(&tw->tp),
+ trace_probe_name(&tw->tp));
+
+ if (tw->type == HW_BREAKPOINT_R)
+ type_str = "r";
+ else if (tw->type == HW_BREAKPOINT_W)
+ type_str = "w";
+ else
+ type_str = "rw";
+
+ if (tw->len == HW_BREAKPOINT_LEN_1)
+ len = 1;
+ else if (tw->len == HW_BREAKPOINT_LEN_2)
+ len = 2;
+ else if (tw->len == HW_BREAKPOINT_LEN_4)
+ len = 4;
+ else
+ len = 8;
+
+ if (tw->symbol) {
+ if (tw->offset)
+ seq_printf(m, " %s@%s%+d:%d", type_str, tw->symbol,
+ tw->offset, len);
+ else
+ seq_printf(m, " %s@%s:%d", type_str, tw->symbol, len);
+#ifdef CONFIG_WPROBE_TRIGGERS
+ } else if (tw->addr == WPROBE_DEFAULT_CLEAR_ADDRESS) {
+ seq_printf(m, " %s@-1:%d", type_str, len);
+#endif
+ } else {
+ seq_printf(m, " %s@0x%lx:%d", type_str, tw->addr, len);
+ }
+
+ for (i = 0; i < tw->tp.nr_args; i++)
+ seq_printf(m, " %s=%s", tw->tp.args[i].name, tw->tp.args[i].comm);
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static enum cpuhp_state wprobe_hp_state __ro_after_init;
+
+static __init int init_wprobe_trace(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "trace/wprobe:online",
+ wprobe_cpu_online, wprobe_cpu_offline);
+ if (ret < 0)
+ return ret;
+
+ wprobe_hp_state = ret;
+
+ ret = dyn_event_register(&trace_wprobe_ops);
+ if (ret)
+ cpuhp_remove_state(wprobe_hp_state);
+
+ return ret;
+}
+fs_initcall(init_wprobe_trace);
+
+#ifdef CONFIG_WPROBE_TRIGGERS
+
+#define SET_WPROBE_STR "set_wprobe"
+#define CLEAR_WPROBE_STR "clear_wprobe"
+#define wprobe_trigger_log_err(file, glob, offs, err) \
+ tracing_log_err((file)->tr, "wprobe_trigger", glob, \
+ trace_probe_err_text, TP_ERR_##err, offs)
+
+struct wprobe_trigger_data {
+ struct rcu_head rcu;
+ struct trace_event_file *file;
+ struct trace_wprobe *tw;
+ int offset;
+ long adjust;
+ const char *field;
+ bool clear;
+};
+
+static bool __wprobe_count_func(struct event_trigger_data *data,
+ unsigned long addr)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long cur_addr;
+
+ if (in_nmi())
+ return false;
+
+ if (atomic_read(&tw->work_pending))
+ return false;
+
+ cur_addr = READ_ONCE(tw->addr);
+
+ if (!wprobe_data->clear) {
+ if (cur_addr != WPROBE_DEFAULT_CLEAR_ADDRESS)
+ return false;
+ if (!trace_wprobe_is_valid_addr(addr, tw->len))
+ return false;
+ } else {
+ if (cur_addr == WPROBE_DEFAULT_CLEAR_ADDRESS)
+ return false;
+ if (wprobe_data->field && cur_addr != addr)
+ return false;
+ }
+
+ return true;
+}
+
+static bool wprobe_count_func(struct event_trigger_data *data,
+ struct trace_buffer *buffer, void *rec,
+ struct ring_buffer_event *event)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long addr = 0;
+
+ if (!data->count)
+ return false;
+
+ if (wprobe_data->field) {
+ if (!rec)
+ return false;
+ addr = *(unsigned long *)((char *)rec + wprobe_data->offset);
+ addr += wprobe_data->adjust;
+ }
+
+ if (!__wprobe_count_func(data, addr)) {
+ atomic_inc(&tw->missed);
+ return false;
+ }
+
+ if (data->count != -1)
+ (data->count)--;
+
+ return true;
+}
+
+static void wprobe_trigger(struct event_trigger_data *data,
+ struct trace_buffer *buffer, void *rec,
+ struct ring_buffer_event *event)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long target_addr, addr = 0, flags;
+
+ if (wprobe_data->field) {
+ if (!rec)
+ return;
+ addr = *(unsigned long *)((char *)rec + wprobe_data->offset);
+ addr += wprobe_data->adjust;
+ }
+
+ raw_spin_lock_irqsave(&tw->lock, flags);
+
+ if (!(data->flags & EVENT_TRIGGER_FL_COUNT)) {
+ if (!__wprobe_count_func(data, addr)) {
+ atomic_inc(&tw->missed);
+ goto out;
+ }
+ }
+
+ target_addr = wprobe_data->clear ? WPROBE_DEFAULT_CLEAR_ADDRESS : addr;
+
+ /*
+ * Try updating the hardware breakpoint on the local CPU first.
+ * If this fails, skip updating tw->addr and do not queue irq_work.
+ * Note that this wprobe will not be traced until tw_addr is updated.
+ * (e.g. NMI happens on this CPU.)
+ */
+ if (trace_wprobe_update_local(tw, target_addr)) {
+ atomic_inc(&tw->missed);
+ goto out;
+ }
+
+ WRITE_ONCE(tw->addr, target_addr);
+ if (!wprobe_data->clear)
+ clear_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+ else
+ set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+
+ /*
+ * For other CPUs, kick irq_work to asynchronously propagate
+ * the new address.
+ */
+ if (num_online_cpus() > 1) {
+ atomic_set(&tw->work_pending, 1);
+ irq_work_queue(&tw->irq_work);
+ }
+
+out:
+ raw_spin_unlock_irqrestore(&tw->lock, flags);
+}
+
+static void free_wprobe_trigger_data(struct wprobe_trigger_data *wprobe_data)
+{
+ if (wprobe_data) {
+ kfree(wprobe_data->field);
+ kfree(wprobe_data);
+ }
+}
+DEFINE_FREE(free_wprobe_trigger_data, struct wprobe_trigger_data *, free_wprobe_trigger_data(_T));
+
+static void free_private_wprobe_trigger_data(struct event_trigger_data *data)
+{
+ free_wprobe_trigger_data(data->private_data);
+}
+
+static int wprobe_trigger_print(struct seq_file *m,
+ struct event_trigger_data *data)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ int missed = atomic_read(&wprobe_data->tw->missed);
+
+ if (wprobe_data->clear) {
+ seq_printf(m, "%s:%s", CLEAR_WPROBE_STR,
+ trace_event_name(wprobe_data->file->event_call));
+ if (wprobe_data->field) {
+ seq_printf(m, ":%s%+ld",
+ wprobe_data->field, wprobe_data->adjust);
+ }
+ } else {
+ seq_printf(m, "%s:%s:%s%+ld", SET_WPROBE_STR,
+ trace_event_name(wprobe_data->file->event_call),
+ wprobe_data->field, wprobe_data->adjust);
+ }
+
+ if (data->count == -1)
+ seq_puts(m, ":unlimited");
+ else
+ seq_printf(m, ":count=%ld", data->count);
+
+ if (data->filter_str)
+ seq_printf(m, " if %s", data->filter_str);
+
+ if (missed)
+ seq_printf(m, " # missed: %d", missed);
+
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static struct wprobe_trigger_data *
+wprobe_trigger_alloc(struct trace_wprobe *tw, struct trace_event_file *file,
+ bool clear)
+{
+ struct wprobe_trigger_data *wprobe_data;
+
+ wprobe_data = kzalloc_obj(*wprobe_data);
+ if (!wprobe_data)
+ return NULL;
+
+ wprobe_data->tw = tw;
+ wprobe_data->clear = clear;
+ wprobe_data->file = file;
+
+ return wprobe_data;
+}
+
+static void wprobe_trigger_free(struct event_trigger_data *data)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+
+ if (WARN_ON_ONCE(data->ref <= 0))
+ return;
+
+ data->ref--;
+ if (!data->ref) {
+ /*
+ * Ensure SOFT_DISABLED flag is restored before clearing soft
+ * mode so that if the user did not explicitly enable this
+ * event, __ftrace_event_enable_disable() unregisters it.
+ */
+ set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+ /* Remove the SOFT_MODE flag */
+ trace_event_enable_disable(wprobe_data->file, 0, 1);
+ trace_event_put_ref(wprobe_data->file->event_call);
+ trigger_data_free(data);
+ }
+}
+
+#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS
+
+static int get_offset_of_field(struct btf *btf, const struct btf_type *type, char *field_name)
+{
+ const struct btf_member *field;
+ const struct btf_type *mtype;
+ int bitoffs = 0;
+ u32 anon_offs;
+ char *next;
+
+ do {
+ next = strchr(field_name, '.');
+ if (next)
+ *next++ = '\0';
+
+ field = btf_find_struct_member(btf, type, field_name, &anon_offs, &mtype);
+ if (IS_ERR_OR_NULL(field))
+ return -ENOENT;
+
+ if (btf_type_kflag(mtype)) {
+ /* Reject bitfield member access */
+ if (BTF_MEMBER_BITFIELD_SIZE(field->offset))
+ return -EINVAL;
+ bitoffs += anon_offs + BTF_MEMBER_BIT_OFFSET(field->offset);
+ } else {
+ bitoffs += anon_offs + field->offset;
+ }
+
+ field_name = next;
+ if (next) {
+ type = btf_type_skip_modifiers(btf, field->type, NULL);
+ if (!type)
+ return -ENOENT;
+ }
+ } while (next);
+ return bitoffs / BITS_PER_BYTE;
+}
+
+/* btf_put(NULL) is acceptable. */
+DEFINE_FREE(btf_put, struct btf *, btf_put(_T))
+
+/* parse typecast: (TYPE[,ASGN])EVENT_FIELD->FIELD[.SUBFIELD...][+-OFFS] and set adjust. */
+static int wprobe_trigger_typecast_parse(char *field_str,
+ struct trace_event_file *file,
+ struct wprobe_trigger_data *wprobe_data,
+ const char *glob)
+{
+ struct btf *btf __free(btf_put) = NULL;
+ char *buf __free(kfree) = NULL;
+ struct ftrace_event_field *field;
+ const struct btf_type *type;
+ char *assign_field;
+ char *event_field;
+ char *type_field;
+ char *type_name;
+ char *offs;
+ long val = 0;
+ int base_offset = field_str - glob;
+ int event_field_offset;
+ int id, adjust;
+
+ buf = kstrdup(field_str, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ type_name = buf + 1;
+ event_field = strchr(type_name, ')');
+ if (!event_field) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_name - buf),
+ DEREF_OPEN_BRACE);
+ return -EINVAL;
+ }
+ *event_field++ = '\0';
+
+ /* Check the optional assign field. */
+ assign_field = strchr(type_name, ',');
+ if (assign_field)
+ *assign_field++ = '\0';
+
+ /* Get the type field name. */
+ type_field = strstr(event_field, "->");
+ if (!type_field) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (event_field - buf),
+ TYPECAST_REQ_FIELD);
+ return -EINVAL;
+ }
+ *type_field = '\0';
+ type_field += 2;
+
+ offs = strpbrk(type_field, "+-");
+ if (offs) {
+ if (kstrtol(offs, 0, &val) < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (offs - buf),
+ BAD_DEREF_OFFS);
+ return -EINVAL;
+ }
+ *offs = '\0';
+ }
+
+ /* find type from BTF */
+ id = bpf_find_btf_id(type_name, BTF_KIND_STRUCT, &btf);
+ if (id < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_name - buf),
+ BAD_BTF_TID);
+ return id;
+ }
+
+ type = btf_type_by_id(btf, id);
+ if (!type) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_name - buf),
+ BAD_BTF_TID);
+ return -EINVAL;
+ }
+
+ adjust = get_offset_of_field(btf, type, type_field);
+ if (adjust < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_field - buf),
+ NO_BTF_FIELD);
+ return adjust;
+ }
+ wprobe_data->adjust = adjust + val;
+
+ if (assign_field) {
+ /* assign_field should be a struct field */
+ adjust = get_offset_of_field(btf, type, assign_field);
+ if (adjust < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (assign_field - buf),
+ NO_BTF_FIELD);
+ return adjust;
+ }
+ wprobe_data->adjust -= adjust;
+ }
+
+ event_field_offset = base_offset + (event_field - buf);
+ field = trace_find_event_field(file->event_call, event_field);
+ if (!field) {
+ wprobe_trigger_log_err(file, glob, event_field_offset, NO_EVENT_FIELD);
+ return -ENOENT;
+ }
+ if (field->size != sizeof(void *)) {
+ wprobe_trigger_log_err(file, glob, event_field_offset, WPROBE_BAD_FIELD);
+ return -ENOEXEC;
+ }
+ wprobe_data->offset = field->offset;
+ wprobe_data->field = kstrdup(event_field, GFP_KERNEL);
+ if (!wprobe_data->field)
+ return -ENOMEM;
+
+ return 0;
+}
+#else
+static int wprobe_trigger_typecast_parse(char *field_str,
+ struct trace_event_file *file,
+ struct wprobe_trigger_data *wprobe_data,
+ const char *glob)
+{
+ wprobe_trigger_log_err(file, glob, field_str - glob, NOSUP_BTFARG);
+ return -EOPNOTSUPP;
+}
+#endif /* CONFIG_PROBE_EVENTS_BTF_ARGS */
+
+static int wprobe_trigger_field_parse(char *field_str, struct trace_event_file *file,
+ struct wprobe_trigger_data *wprobe_data,
+ const char *glob)
+{
+ struct ftrace_event_field *field;
+ char *offs;
+
+ if (field_str[0] == '(')
+ return wprobe_trigger_typecast_parse(field_str, file, wprobe_data, glob);
+
+ offs = strpbrk(field_str, "+-");
+ if (offs) {
+ long val;
+
+ if (kstrtol(offs, 0, &val) < 0) {
+ wprobe_trigger_log_err(file, glob, offs - glob, BAD_DEREF_OFFS);
+ return -EINVAL;
+ }
+ wprobe_data->adjust = val;
+ *offs = '\0';
+ }
+
+ field = trace_find_event_field(file->event_call, field_str);
+ if (!field) {
+ wprobe_trigger_log_err(file, glob, field_str - glob, NO_EVENT_FIELD);
+ return -ENOENT;
+ }
+ if (field->size != sizeof(void *)) {
+ wprobe_trigger_log_err(file, glob, field_str - glob, WPROBE_BAD_FIELD);
+ return -ENOEXEC;
+ }
+ wprobe_data->offset = field->offset;
+ wprobe_data->field = kstrdup(field_str, GFP_KERNEL);
+ if (!wprobe_data->field)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops,
+ struct trace_event_file *file,
+ char *glob, char *cmd,
+ char *param_and_filter)
+{
+ /*
+ * set_wprobe:EVENT:FIELD[+OFFS]
+ * clear_wprobe:EVENT[:FIELD[+OFFS]]
+ */
+ struct wprobe_trigger_data *wprobe_data = NULL;
+ struct event_trigger_data *trigger_data = NULL;
+ struct trace_event_file *wprobe_file;
+ char *event_str, *comment;
+ struct trace_array *tr = file->tr;
+ bool remove, clear = false;
+ unsigned long orig_addr = 0;
+ struct trace_wprobe *tw;
+ char *param, *filter;
+ int ret;
+
+ remove = event_trigger_check_remove(glob);
+
+ if (!strcmp(cmd, CLEAR_WPROBE_STR))
+ clear = true;
+
+ if (param_and_filter) {
+ /* Recover original trigger string to show the error log correctly. */
+ if (*(param_and_filter - 1) == '\0')
+ *(param_and_filter - 1) = ':';
+ comment = strchr(param_and_filter, '#');
+ if (comment)
+ *comment = '\0';
+ }
+
+ if (event_trigger_empty_param(param_and_filter)) {
+ wprobe_trigger_log_err(file, glob, strlen(cmd) + 1, WPROBE_NOT_FOUND);
+ return -EINVAL;
+ }
+
+ ret = event_trigger_separate_filter(param_and_filter, ¶m, &filter, true);
+ if (ret)
+ return ret;
+
+ if (file->event_call->flags & TRACE_EVENT_FL_KPROBE) {
+ wprobe_trigger_log_err(file, glob, 0, WPROBE_ON_KPROBE);
+ return -EOPNOTSUPP;
+ }
+
+ event_str = strsep(¶m, ":");
+
+ /* Find target wprobe */
+ tw = find_trace_wprobe(event_str, WPROBE_EVENT_SYSTEM);
+ if (!tw) {
+ wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND);
+ return -ENOENT;
+ }
+ /* The target wprobe must not be used (unless clear) */
+ if (!remove && !clear && trace_probe_is_enabled(&tw->tp)) {
+ wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_BUSY);
+ return -EBUSY;
+ }
+
+ wprobe_file = find_event_file(tr, WPROBE_EVENT_SYSTEM, event_str);
+ if (!wprobe_file) {
+ wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND);
+ return -EINVAL;
+ }
+
+ wprobe_data = wprobe_trigger_alloc(tw, wprobe_file, clear);
+ if (!wprobe_data)
+ return -ENOMEM;
+
+ /* clear_wprobe does not need field, but can have optional field. */
+ if (!clear) {
+ char *field_str = strsep(¶m, ":");
+
+ if (!field_str) {
+ wprobe_trigger_log_err(file, glob, strlen(glob), WPROBE_NEED_FIELD);
+ ret = -EINVAL;
+ goto out_free;
+ }
+ ret = wprobe_trigger_field_parse(field_str, file, wprobe_data, glob);
+ if (ret < 0)
+ goto out_free;
+ } else if (param && (isalpha(param[0]) || param[0] == '_' || param[0] == '(')) {
+ if (strncmp(param, "count=", 6) != 0 &&
+ (strcmp(param, "unlimited") != 0 ||
+ trace_find_event_field(file->event_call, "unlimited"))) {
+ char *field_str = strsep(¶m, ":");
+
+ ret = wprobe_trigger_field_parse(field_str, file, wprobe_data, glob);
+ if (ret < 0)
+ goto out_free;
+ }
+ }
+
+ if (param) {
+ if (!strcmp(param, "unlimited"))
+ param = NULL;
+ else if (!strncmp(param, "count=", 6))
+ param += 6;
+ }
+
+ trigger_data = trigger_data_alloc(cmd_ops, cmd, param, wprobe_data);
+ if (!trigger_data) {
+ ret = -ENOMEM;
+ goto out_free;
+ }
+
+ trigger_data->private_data_free = free_private_wprobe_trigger_data;
+
+ if (remove) {
+ event_trigger_unregister(cmd_ops, file, glob+1, trigger_data);
+ trigger_data_free(trigger_data);
+ return 0;
+ }
+
+ /* Up the trigger_data count to make sure nothing frees it on failure */
+ event_trigger_init(trigger_data);
+
+ ret = event_trigger_parse_num(param, trigger_data);
+ if (ret) {
+ wprobe_trigger_log_err(file, glob, param - glob, BAD_IMM);
+ goto out_free_trigger;
+ }
+
+ ret = event_trigger_set_filter(cmd_ops, file, filter, trigger_data);
+ if (ret < 0)
+ goto out_free_trigger;
+
+ /* Soft-enable (register) wprobe event on WPROBE_DEFAULT_CLEAR_ADDRESS */
+ if (!trace_event_try_get_ref(wprobe_file->event_call)) {
+ ret = -ENODEV;
+ goto out_free_trigger;
+ }
+
+ if (!clear) {
+ orig_addr = tw->addr;
+ WRITE_ONCE(tw->addr, WPROBE_DEFAULT_CLEAR_ADDRESS);
+ }
+
+ ret = trace_event_enable_disable(wprobe_file, 1, 1);
+ if (ret < 0) {
+ if (!clear)
+ WRITE_ONCE(tw->addr, orig_addr);
+ goto out_put;
+ }
+
+ ret = event_trigger_register(cmd_ops, file, glob, trigger_data);
+ if (ret)
+ goto out_disable;
+
+ /* Balance the event_trigger_init() at registration start */
+ event_trigger_free(trigger_data);
+ return 0;
+
+out_disable:
+ if (!clear)
+ WRITE_ONCE(tw->addr, orig_addr);
+ trace_event_enable_disable(wprobe_file, 0, 1);
+out_put:
+ trace_event_put_ref(wprobe_file->event_call);
+out_free_trigger:
+ event_trigger_reset_filter(cmd_ops, trigger_data);
+ trigger_data_free(trigger_data);
+ return ret;
+
+out_free:
+ free_wprobe_trigger_data(wprobe_data);
+ return ret;
+}
+
+/* Return event_trigger_data if there is a trigger which points the same wprobe */
+static struct event_trigger_data *
+wprobe_trigger_find_same(struct event_trigger_data *test,
+ struct trace_event_file *file)
+{
+ struct wprobe_trigger_data *test_wprobe_data = test->private_data;
+ struct wprobe_trigger_data *wprobe_data;
+ struct event_trigger_data *iter;
+
+ list_for_each_entry(iter, &file->triggers, list) {
+ wprobe_data = iter->private_data;
+ if (!wprobe_data ||
+ iter->cmd_ops->trigger_type !=
+ test->cmd_ops->trigger_type)
+ continue;
+ if (wprobe_data->tw == test_wprobe_data->tw &&
+ wprobe_data->clear == test_wprobe_data->clear)
+ return iter;
+ }
+ return NULL;
+}
+
+static int wprobe_register_trigger(char *glob,
+ struct event_trigger_data *data,
+ struct trace_event_file *file)
+{
+ int ret = 0;
+
+ lockdep_assert_held(&event_mutex);
+
+ /* The same wprobe is not accept on the same file (event) */
+ if (wprobe_trigger_find_same(data, file))
+ return -EEXIST;
+
+ if (data->cmd_ops->init) {
+ ret = data->cmd_ops->init(data);
+ if (ret < 0)
+ return ret;
+ }
+
+ list_add_rcu(&data->list, &file->triggers);
+
+ update_cond_flag(file);
+ ret = trace_event_trigger_enable_disable(file, 1);
+ if (ret < 0) {
+ list_del_rcu(&data->list);
+ update_cond_flag(file);
+ tracepoint_synchronize_unregister();
+ if (data->cmd_ops->free)
+ data->cmd_ops->free(data);
+ }
+ return ret;
+}
+
+static void wprobe_unregister_trigger(char *glob,
+ struct event_trigger_data *test,
+ struct trace_event_file *file)
+{
+ struct event_trigger_data *data;
+
+ lockdep_assert_held(&event_mutex);
+
+ data = wprobe_trigger_find_same(test, file);
+ if (!data)
+ return;
+
+ list_del_rcu(&data->list);
+ trace_event_trigger_enable_disable(file, 0);
+ update_cond_flag(file);
+ tracepoint_synchronize_unregister();
+ if (data->cmd_ops->free)
+ data->cmd_ops->free(data);
+}
+
+static struct event_command trigger_wprobe_set_cmd = {
+ .name = SET_WPROBE_STR,
+ .trigger_type = ETT_EVENT_WPROBE,
+ /* This triggers after when the event is recorded. */
+ .flags = EVENT_CMD_FL_NEEDS_REC,
+ .parse = wprobe_trigger_cmd_parse,
+ .reg = wprobe_register_trigger,
+ .unreg = wprobe_unregister_trigger,
+ .set_filter = set_trigger_filter,
+ .trigger = wprobe_trigger,
+ .count_func = wprobe_count_func,
+ .print = wprobe_trigger_print,
+ .init = event_trigger_init,
+ .free = wprobe_trigger_free,
+};
+
+static struct event_command trigger_wprobe_clear_cmd = {
+ .name = CLEAR_WPROBE_STR,
+ .trigger_type = ETT_EVENT_WPROBE,
+ /* This triggers after when the event is recorded. */
+ .flags = EVENT_CMD_FL_NEEDS_REC,
+ .parse = wprobe_trigger_cmd_parse,
+ .reg = wprobe_register_trigger,
+ .unreg = wprobe_unregister_trigger,
+ .set_filter = set_trigger_filter,
+ .trigger = wprobe_trigger,
+ .count_func = wprobe_count_func,
+ .print = wprobe_trigger_print,
+ .init = event_trigger_init,
+ .free = wprobe_trigger_free,
+};
+
+static __init int init_trigger_wprobe_cmds(void)
+{
+ int ret;
+
+ ret = register_event_command(&trigger_wprobe_set_cmd);
+ if (WARN_ON(ret < 0))
+ return ret;
+ ret = register_event_command(&trigger_wprobe_clear_cmd);
+ if (WARN_ON(ret < 0))
+ unregister_event_command(&trigger_wprobe_set_cmd);
+
+ return ret;
+}
+fs_initcall(init_trigger_wprobe_cmds);
+#endif /* CONFIG_WPROBE_TRIGGERS */
diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config
index 544de0db5f58f..f067874902ed7 100644
--- a/tools/testing/selftests/ftrace/config
+++ b/tools/testing/selftests/ftrace/config
@@ -1,5 +1,6 @@
CONFIG_BPF_SYSCALL=y
CONFIG_DEBUG_INFO_BTF=y
+CONFIG_DEBUG_INFO_BTF_MODULES=y
CONFIG_DEBUG_INFO_DWARF4=y
CONFIG_EPROBE_EVENTS=y
CONFIG_FPROBE=y
@@ -27,3 +28,5 @@ CONFIG_STACK_TRACER=y
CONFIG_TRACER_SNAPSHOT=y
CONFIG_UPROBES=y
CONFIG_UPROBE_EVENTS=y
+CONFIG_WPROBE_EVENTS=y
+CONFIG_WPROBE_TRIGGERS=y
diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
new file mode 100644
index 0000000000000..3c622fb07e7f9
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
@@ -0,0 +1,66 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Generic dynamic event - add/remove wprobe events
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README
+
+echo 0 > events/enable
+echo > dynamic_events
+
+# Use jiffies as a variable that is frequently written to.
+TARGET=jiffies
+
+echo "w:my_wprobe w@$TARGET" >> dynamic_events
+
+if ! grep -q my_wprobe dynamic_events; then
+ echo "Failed to create wprobe event"
+ exit_fail
+fi
+
+if [ ! -d events/wprobes/my_wprobe ]; then
+ echo "Failed to create wprobe event directory"
+ exit_fail
+fi
+
+if ! echo 1 > events/wprobes/my_wprobe/enable; then
+ echo "Failed to enable wprobe (hardware watchpoints may be exhausted)"
+ exit_unresolved
+fi
+
+# Check if the event is enabled
+if ! grep -q 1 events/wprobes/my_wprobe/enable; then
+ echo "Failed to enable wprobe event"
+ exit_fail
+fi
+
+# Let some time pass to trigger the breakpoint
+sleep 1
+
+# Check if we got any trace output
+if ! grep -q my_wprobe trace; then
+ echo "wprobe event was not triggered"
+ exit_fail
+fi
+
+echo 0 > events/wprobes/my_wprobe/enable
+
+# Check if the event is disabled
+if ! grep -q 0 events/wprobes/my_wprobe/enable; then
+ echo "Failed to disable wprobe event"
+ exit_fail
+fi
+
+echo "-:my_wprobe" >> dynamic_events
+
+if grep -q my_wprobe dynamic_events; then
+ echo "Failed to remove wprobe event"
+ exit_fail
+fi
+
+if [ -d events/wprobes/my_wprobe ]; then
+ echo "Failed to remove wprobe event directory"
+ exit_fail
+fi
+
+clear_trace
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
new file mode 100644
index 0000000000000..e0df58fe7aa3e
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
@@ -0,0 +1,23 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Watch probe event parser error log check
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README
+
+check_error() { # command-with-error-pos-by-^
+ ftrace_errlog_check 'wprobe' "$1" 'dynamic_events'
+}
+
+check_error '^w' # NO_ARG_BODY
+check_error 'w^10 w@jiffies' # WPROBE_NO_MAXACT
+check_error 'w ^symbol' # BAD_ACCESS_FMT
+check_error 'w ^a@symbol' # BAD_ACCESS_TYPE
+check_error 'w w@^symbol' # BAD_ACCESS_ADDR
+check_error 'w w@^0:4' # BAD_ACCESS_ADDR
+check_error 'w w@jiffies^+offset' # BAD_PROBE_ADDR
+check_error 'w w@jiffies:^100' # BAD_ACCESS_LEN
+check_error 'w w@jiffies ^$arg1' # BAD_VAR
+check_error 'w w@jiffies ^$retval' # BAD_VAR
+check_error 'w w@jiffies ^$stack' # BAD_VAR
+check_error 'w w@jiffies ^%ax' # BAD_VAR
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
new file mode 100644
index 0000000000000..8bc8ca02a712e
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
@@ -0,0 +1,85 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test set_wprobe trigger with BTF struct offset
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event watching 8 bytes" ;:
+echo 'w:watch rw@-1:8 address=$addr value=$value' >> dynamic_events
+
+:;: "Add fprobe event for sample_timer_cb" ;:
+# sample_timer_cb(struct timer_list *t)
+# container_of(t, struct foo_timer_data, timer)
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+
+:;: "Set set_wprobe trigger using BTF struct offset resolution" ;:
+# Syntax: set_wprobe:WPEVENT:(STRUCT,FIELD)EVENT_FIELD->MEMBER
+# (foo_timer_data,timer) is the BTF struct type and field name
+# timer->expires is the struct member whose offset is resolved automatically via BTF
+echo 'set_wprobe:watch:(foo_timer_data,timer)timer->timer.expires' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate
+sleep 3
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+# Since we don't know actual offset of timer->expires in foo_timer_data, we use reset_trigger_file
+reset_trigger_file events/fprobes/testevent/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
new file mode 100644
index 0000000000000..3d4c9fd6b8e23
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
@@ -0,0 +1,85 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger with BTF typecast fetchargs
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event with BTF typecast fetchargs" ;:
+# (foo_timer_data,timer)$addr->counter reads counter from struct foo_timer_data via BTF typecast
+echo 'w:watch rw@-1:8 address=$addr counter=(foo_timer_data,timer)$addr->counter' >> dynamic_events
+
+:;: "Check the wprobe event is registered with counter field" ;:
+if ! grep -q "counter" dynamic_events; then
+ fail "Failed to register counter field in dynamic_events"
+fi
+
+:;: "Add fprobe event for sample_timer_cb" ;:
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+
+:;: "Set set_wprobe trigger on testevent" ;:
+echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate
+sleep 3
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc
new file mode 100644
index 0000000000000..f82b13d813db1
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc
@@ -0,0 +1,150 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger across cpu hotplug
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+if ! which nproc >/dev/null 2>&1 ; then
+ nproc() {
+ ls -d /sys/devices/system/cpu/cpu[0-9]* | wc -l
+ }
+fi
+
+NP=`nproc`
+if [ $NP -le 1 ] ; then
+ echo "We cannot test cpu hotplug in UP environment"
+ exit_unresolved
+fi
+
+# Find an online CPU that can be offlined
+for i in /sys/devices/system/cpu/cpu[1-9]*; do
+ if [ -f $i/online ] && [ "$(cat $i/online)" = "1" ]; then
+ cpu=$i
+ break
+ fi
+done
+
+if [ -z "$cpu" ]; then
+ echo "We cannot test cpu hotplug without a hotpluggable online cpu"
+ exit_unresolved
+fi
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ [ -n "$cpu" ] && [ -f "$cpu/online" ] && echo 1 > $cpu/online || true
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then
+ reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event used by trigger" ;:
+echo 'w:watch rw@-1:8 address=$addr value=$value' > dynamic_events
+
+:;: "Add events for triggering wprobe" ;:
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+echo 1 >> events/sample-trace/foo_bar_with_fn/enable
+
+:;: "Set set_wprobe trigger on testevent" ;:
+echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+:;: "Take target CPU offline" ;:
+echo 0 > $cpu/online
+sleep 1
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate while CPU is offline
+sleep 2
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint while CPU is offline"
+fi
+
+:;: "Bring target CPU back online" ;:
+echo 1 > $cpu/online
+sleep 1
+
+# Clear trace and verify that watchpoint triggers after CPU is onlined
+clear_trace
+sleep 2
+
+:;: "Verify watchpoint triggers after CPU came back online" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint after CPU was onlined"
+fi
+
+:;: "Set clear_wprobe trigger on foo_bar_with_fn" ;:
+echo 'clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to set clear_wprobe trigger"
+fi
+
+# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpoint
+echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+
+# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate watchpoint
+sleep 2
+
+# Clear trace and wait to ensure no new watchpoint events are generated
+clear_trace
+sleep 1
+
+:;: "Ensure clear_wprobe trigger deactivated the watchpoint" ;:
+if grep -q watch trace; then
+ fail "Failed to clear watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+echo '!clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to remove clear_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
new file mode 100644
index 0000000000000..dd1ce4baa62be
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
@@ -0,0 +1,37 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger syntax errors
+# requires: dynamic_events error_log "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger
+
+check_error() { # command-with-error-pos-by-^
+ ftrace_errlog_check "wprobe_trigger" "$1" "events/fprobes/testevent/trigger"
+}
+
+TARGET_FUNC=$(grep -m 1 -E -w "[tT] (vfs_read|do_sys_openat2|do_sys_open)" /proc/kallsyms | awk '{print $3}')
+if [ -z "$TARGET_FUNC" ]; then
+ echo "UNRESOLVED: target function not found"
+ exit_unresolved
+fi
+
+# Add a dummy fprobe event to attach triggers to
+echo "f:fprobes/testevent $TARGET_FUNC dfd=\$arg1" > dynamic_events
+
+# Add a target wprobe event
+echo 'w:watch rw@-1:8' >> dynamic_events
+
+# Test errors on trigger syntax
+check_error 'set_wprobe:^non_exist_wprobe:dfd' # WPROBE_NOT_FOUND
+check_error 'set_wprobe:^' # WPROBE_NOT_FOUND
+check_error 'set_wprobe:watch^' # WPROBE_NEED_FIELD
+check_error 'set_wprobe:watch:^non_exist_field' # NO_EVENT_FIELD
+
+# Enable target wprobe event and test WPROBE_BUSY error
+echo 1 > events/wprobes/watch/enable
+check_error 'set_wprobe:^watch:dfd' # WPROBE_BUSY
+echo 0 > events/wprobes/watch/enable
+
+# Cleanup
+echo '-:watch' >> dynamic_events
+echo '-:testevent' >> dynamic_events
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
new file mode 100644
index 0000000000000..043e1c915ccf1
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
@@ -0,0 +1,107 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then
+ reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event used by trigger" ;:
+echo 'w:watch rw@-1:8 address=$addr value=$value' > dynamic_events
+
+:;: "Add events for triggering wprobe" ;:
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+echo 1 >> events/sample-trace/foo_bar_with_fn/enable
+
+:;: "Set set_wprobe trigger on testevent" ;:
+echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate
+sleep 2
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint"
+fi
+
+:;: "Set clear_wprobe trigger on foo_bar_with_fn" ;:
+echo 'clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to set clear_wprobe trigger"
+fi
+
+# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpoint
+echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+
+# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate watchpoint
+sleep 2
+
+# Clear trace and wait to ensure no new watchpoint events are generated
+clear_trace
+sleep 1
+
+:;: "Ensure clear_wprobe trigger deactivated the watchpoint" ;:
+if grep -q watch trace; then
+ fail "Failed to clear watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+echo '!clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to remove clear_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 65eb26a0520d8..7779d500fe774 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -6476,11 +6476,23 @@ static unsigned long kvm_guest_get_ip(void)
return kvm_arch_vcpu_get_ip(vcpu);
}
+static bool kvm_guest_in_guest(void)
+{
+ struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+
+ if (!vcpu)
+ return false;
+
+ return READ_ONCE(vcpu->mode) == IN_GUEST_MODE ||
+ READ_ONCE(vcpu->mode) == EXITING_GUEST_MODE;
+}
+
static struct perf_guest_info_callbacks kvm_guest_cbs = {
.state = kvm_guest_state,
.get_ip = kvm_guest_get_ip,
.handle_intel_pt_intr = NULL,
.handle_mediated_pmi = NULL,
+ .in_guest = kvm_guest_in_guest,
};
void __kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void),
|
| 3/2 | 2026/09/24 05:44 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 46767 output: 23 thoughts: 1491 Thoughts: **Fuzzing Recommendation: Worthy** Okay, let's break this down. My initial task is to decide if this patch series warrants fuzzing with syzkaller. First, I need to understand what this code is actually doing. Alright, so the patch introduces "Watchpoint probe (wprobe) Event Tracing." Looks like it's adding a new tracing mechanism using hardware breakpoints. The patch touches a lot of files, including documentation, x86 architecture-specific code (debug register handling, hardware breakpoint management, trap/NMI/MCE handlers), kernel event handling, and a new `trace_wprobe.c` file which seems to contain the core implementation. There are a bunch of modifications to the `tracefs` interface. Now, the core triage philosophy needs to be considered. The primary factor is reachability. Can this code *actually* be executed in a standard virtualized environment like GCE or QEMU? And the answer is yes! Tracing mechanisms are accessible through `tracefs`. We can write commands to `/sys/kernel/tracing` as root (which syzkaller has) using `dynamic_events` and the trigger interface. This means the code is reachable. The patch also touches core subsystems: tracing, perf hw_breakpoints, x86 debugreg/traps. So, according to the "WHEN TO RETURN WorthFuzzing=true" criteria, this *definitely* looks like a go. This is more than just a minor change; we have a new dynamic event system in `trace_wprobe.c` (~1668 lines!) plus trigger integration, *and* modifications to the x86 hardware breakpoint and trap/MCE handling. This is a significant addition to tracing capabilities. Next, I need to check the Kconfig dependencies. This is important to determine what configurations need to be set to enable the new functionality. Specifically, the patch enables `WPROBE_EVENTS`, `WPROBE_TRIGGERS`, and `PROBE_EVENTS_BTF_ARGS`. These depend on `TRACING`, `HAVE_HW_BREAKPOINT`, `HAVE_POST_BREAKPOINT_HOOK`, and `HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR`. Quick check of the x86 Kconfig confirms that on x86, the required `HAVE_*` options are automatically selected. The `TRACING` option is selected by various tracers or can be enabled with `CONFIG_TRACING`. The presence of `CONFIG_PROBE_EVENTS_BTF_ARGS` indicates the potential for more advanced argument handling. **Conclusion:** **Fuzzing is warranted.** This patch series introduces new, reachable functionality that interacts with critical subsystems, and the necessary configurations are readily available on x86. |
| 4/2 | 2026/09/24 05:45 | tool | grepper |
0mArgs:{
"Expression": "PROBE_EVENTS_BTF_ARGS"
}Results: {
"Output": "kernel/trace/Kconfig=290=config FUNCTION_TRACE_ARGS\nkernel/trace/Kconfig-291- bool\nkernel/trace/Kconfig:292:\tdepends on PROBE_EVENTS_BTF_ARGS\nkernel/trace/Kconfig-293-\tdefault y\n--\nkernel/trace/Kconfig=756=config FPROBE_EVENTS\n--\nkernel/trace/Kconfig-769-\nkernel/trace/Kconfig:770:config PROBE_EVENTS_BTF_ARGS\nkernel/trace/Kconfig-771-\tdepends on HAVE_FUNCTION_ARG_ACCESS_API\n--\nkernel/trace/Makefile=121=obj-$(CONFIG_PROBE_EVENTS) += trace_probe.o\nkernel/trace/Makefile:122:obj-$(CONFIG_PROBE_EVENTS_BTF_ARGS) += trace_btf.o\nkernel/trace/Makefile-123-obj-$(CONFIG_UPROBE_EVENTS) += trace_uprobe.o\n--\nkernel/trace/trace.c=4183=static const char readme_msg[] =\n--\nkernel/trace/trace.c-4326-\t\"\\t $stack\u003cindex\u003e, $stack, $retval, $comm, $arg\u003cN\u003e, $current\\n\"\nkernel/trace/trace.c:4327:#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS\nkernel/trace/trace.c-4328-\t\"\\t [(structname[,field])]\u003cargname\u003e[-\u003efield[-\u003efield|.field...]],\\n\"\n--\nkernel/trace/trace_probe.c=359=static int parse_this_cpu(char *arg, struct traceprobe_parse_context *ctx)\n--\nkernel/trace/trace_probe.c-401-\nkernel/trace/trace_probe.c:402:#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS\nkernel/trace/trace_probe.c-403-\n--\nkernel/trace/trace_probe.c=969=static int handle_typecast(char *arg, struct traceprobe_parse_context *ctx)\n--\nkernel/trace/trace_probe.c-1072-\nkernel/trace/trace_probe.c:1073:#else /* !CONFIG_PROBE_EVENTS_BTF_ARGS */\nkernel/trace/trace_probe.c-1074-\n--\nkernel/trace/trace_probe.c=1128=static int handle_typecast(char *arg, struct traceprobe_parse_context *ctx)\n--\nkernel/trace/trace_probe.c-1133-\nkernel/trace/trace_probe.c:1134:#endif /* CONFIG_PROBE_EVENTS_BTF_ARGS */\nkernel/trace/trace_probe.c-1135-\n--\nkernel/trace/trace_probe.c=1274=static int parse_probe_var_retval(char *orig_arg,\n--\nkernel/trace/trace_probe.c-1285-\tif (!(ctx-\u003eflags \u0026 TPARG_FL_KERNEL) ||\nkernel/trace/trace_probe.c:1286:\t !IS_ENABLED(CONFIG_PROBE_EVENTS_BTF_ARGS)) {\nkernel/trace/trace_probe.c-1287-\t\tcode-\u003eop = FETCH_OP_RETVAL;\n--\nkernel/trace/trace_probe.c=1325=static int parse_probe_var_current(char *orig_arg, char *arg,\n--\nkernel/trace/trace_probe.c-1337-\targ += strlen(\"current\");\nkernel/trace/trace_probe.c:1338:\tif (*arg == '-' \u0026\u0026 IS_ENABLED(CONFIG_PROBE_EVENTS_BTF_ARGS))\nkernel/trace/trace_probe.c-1339-\t\treturn parse_btf_arg(orig_arg, pcode, end, ctx);\n--\nkernel/trace/trace_probe.c=1937=static int finalize_fetch_insn(struct fetch_insn *code,\n--\nkernel/trace/trace_probe.c-2022-\t\t}\nkernel/trace/trace_probe.c:2023:\t} else if (IS_ENABLED(CONFIG_PROBE_EVENTS_BTF_ARGS) \u0026\u0026\nkernel/trace/trace_probe.c-2024-\t\t ctx-\u003elast_type) {\n--\nkernel/trace/trace_probe.c=2056=static int traceprobe_parse_probe_arg_body(const char *argv, ssize_t *size,\n--\nkernel/trace/trace_probe.c-2095-\t/* Update storing type if BTF is available */\nkernel/trace/trace_probe.c:2096:\tif (IS_ENABLED(CONFIG_PROBE_EVENTS_BTF_ARGS) \u0026\u0026\nkernel/trace/trace_probe.c-2097-\t ctx-\u003elast_type) {\n--\nkernel/trace/trace_probe.c=2172=static char *generate_probe_arg_name(const char *arg, int idx)\n--\nkernel/trace/trace_probe.c-2180-\t */\nkernel/trace/trace_probe.c:2181:\tif (IS_ENABLED(CONFIG_PROBE_EVENTS_BTF_ARGS)) {\nkernel/trace/trace_probe.c-2182-\t\tend = strchr(arg, ':');\n--\nkernel/trace/trace_wprobe.c=1147=static void wprobe_trigger_free(struct event_trigger_data *data)\n--\nkernel/trace/trace_wprobe.c-1168-\nkernel/trace/trace_wprobe.c:1169:#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS\nkernel/trace/trace_wprobe.c-1170-\n--\nkernel/trace/trace_wprobe.c=1327=static int wprobe_trigger_typecast_parse(char *field_str,\n--\nkernel/trace/trace_wprobe.c-1334-}\nkernel/trace/trace_wprobe.c:1335:#endif /* CONFIG_PROBE_EVENTS_BTF_ARGS */\nkernel/trace/trace_wprobe.c-1336-\n--\ntools/testing/selftests/ftrace/config=20=CONFIG_PREEMPT_TRACER=y\ntools/testing/selftests/ftrace/config:21:CONFIG_PROBE_EVENTS_BTF_ARGS=y\ntools/testing/selftests/ftrace/config-22-CONFIG_SAMPLES=y\n"
} |
| 5/2 | 2026/09/24 05:45 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 50487 output: 34 thoughts: 27 Thoughts: Here's my analysis of the situation: **Investigating `PROBE_EVENTS_BTF_ARGS` in the Kernel Configuration** Okay, so I need to understand what `PROBE_EVENTS_BTF_ARGS` is all about. My initial thought is to go straight to the source – the `Kconfig` file within the kernel's tracing subsystem. This is where I'd expect to find the definitive definition and any associated context or dependencies for this configuration option. I'm assuming this Kconfig entry will give me the most fundamental information. Let's see what we can find. I anticipate a concise explanation, possibly some associated dependencies, and potentially some default values or examples. From there, I can begin to unpack what it enables, disables or configures in the kernel. |
| 6/2 | 2026/09/24 05:45 | tool | read-file |
0mArgs:{
"File": "kernel/trace/Kconfig",
"FirstLine": 765,
"LineCount": 20
}Results: {
"Contents": " 765:\t\t This allows user to add tracing events on the function entry and\n 766:\t\t exit via ftrace interface. The syntax is same as the kprobe events\n 767:\t\t and the kprobe events on function entry and exit will be\n 768:\t\t transparently converted to this fprobe events.\n 769:\t\n 770:\tconfig PROBE_EVENTS_BTF_ARGS\n 771:\t\tdepends on HAVE_FUNCTION_ARG_ACCESS_API\n 772:\t\tdepends on FPROBE_EVENTS || KPROBE_EVENTS\n 773:\t\tdepends on DEBUG_INFO_BTF \u0026\u0026 BPF_SYSCALL\n 774:\t\tbool \"Support BTF function arguments for probe events\"\n 775:\t\tdefault y\n 776:\t\thelp\n 777:\t\t The user can specify the arguments of the probe event using the names\n 778:\t\t of the arguments of the probed function, when the probe location is a\n 779:\t\t kernel function entry or a tracepoint.\n 780:\t\t This is available only if BTF (BPF Type Format) support is enabled.\n 781:\t\n 782:\tconfig PROBE_EVENTS_DUMP_FETCHARG\n 783:\t\tbool \"Dump of dynamic probe event fetch-arguments\"\n 784:\t\tdepends on PROBE_EVENTS\n"
} |
| 7/2 | 2026/09/24 05:45 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 51067 output: 34 thoughts: 21 Thoughts: Here's my summary, as though I'm thinking through this process: **Diving into Kernel Trace Configuration** Okay, so I'm digging into kernel tracing, specifically looking at the `WPROBE_EVENTS` option within the `kernel/trace/Kconfig` file. This is crucial; `Kconfig` files are the backbone of the kernel's build configuration system. Understanding this option is fundamental to enabling and configuring specific tracepoints, which is how we gather detailed runtime information about kernel behavior. `WPROBE_EVENTS`, in particular, likely points to a set of tracepoints that are associated with probe events; the "W" suggests these are related to write or modification operations. I need to figure out the exact implications. Is it enabling a broader category of tracepoints or a specific subset? What are the dependencies? This could be a critical setting for debugging, profiling, and understanding kernel performance bottlenecks, so I need to understand its scope and impact completely. It probably enables a category of tracepoints that need to be understood in the context of the larger tracer system. Time to delve into the code! |
| 8/2 | 2026/09/24 05:45 | tool | read-file |
0mArgs:{
"File": "kernel/trace/Kconfig",
"FirstLine": 860,
"LineCount": 35
}Results: {
"Contents": " 860:\t\t events. It can be used to limit what fields are recorded in\n 861:\t\t an event or even dereference a field of an event. It can\n 862:\t\t convert the type of an event field. For example, turn an\n 863:\t\t address into a string.\n 864:\t\n 865:\tconfig WPROBE_EVENTS\n 866:\t\tbool \"Enable wprobe-based dynamic events\"\n 867:\t\tdepends on TRACING\n 868:\t\tdepends on HAVE_HW_BREAKPOINT\n 869:\t\tdepends on HAVE_POST_BREAKPOINT_HOOK\n 870:\t\tselect PROBE_EVENTS\n 871:\t\tselect DYNAMIC_EVENTS\n 872:\t\thelp\n 873:\t\t This allows the user to add watchpoint tracing events based on\n 874:\t\t hardware breakpoints on the fly via the ftrace interface.\n 875:\t\n 876:\t\t Those events can be inserted wherever hardware breakpoints can be\n 877:\t\t set, and record accessed memory address and values.\n 878:\t\n 879:\tconfig WPROBE_TRIGGERS\n 880:\t\tdepends on WPROBE_EVENTS\n 881:\t\tdepends on HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR\n 882:\t\tbool\n 883:\t\tdefault y\n 884:\t\thelp\n 885:\t\t This adds an event trigger which will set the wprobe on a specific\n 886:\t\t field of an event. This allows user to trace the memory access of\n 887:\t\t an address pointed by the event field.\n 888:\t\n 889:\tconfig BPF_EVENTS\n 890:\t\tdepends on BPF_SYSCALL\n 891:\t\tdepends on (KPROBE_EVENTS || UPROBE_EVENTS) \u0026\u0026 PERF_EVENTS\n 892:\t\tbool\n 893:\t\tdefault y\n 894:\t\thelp\n"
} |
| 9/2 | 2026/09/24 05:45 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 51866 output: 258 thoughts: 2443 Thoughts: **Analysis of Watchpoint Probe (Wprobe) and Hardware Breakpoint Feature** Alright, let's break this down. My initial assessment is this: we're dealing with a patch that introduces a new dynamic tracing capability, `WPROBE_EVENTS`, and triggers, `WPROBE_TRIGGERS`, which are essentially watchpoints on kernel memory addresses. This, in turn, heavily modifies the existing x86 hardware breakpoint handling. The introduction of `PROBE_EVENTS_BTF_ARGS` for BTF typecast support further complicates the picture. Given the nature of this change, dealing with new dynamic events, breakpoint alteration, and user-space interaction through `tracefs`, fuzzing this is absolutely justified, and `WorthFuzzing` is unequivocally `true`. The primary reason is the high likelihood of race conditions. This patch touches on several areas that have the potential for concurrency issues, including event triggers, IRQ work, SMP function calls, CPU hotplug, MCE, and NMI handling. The interactions between these subsystems create fertile ground for race conditions, deadlocks, state inconsistencies, or assertion failures—all ripe for fuzzing targets. Now, let's identify the specific functions to target for fuzzing—the `FocusSymbols`. The goal is to focus on entrypoints, and the functions that specifically implement the logic being added or altered. I need to distinguish between static inlines (which can't be targeted by symbol coverage), and those with distinct symbols. After reviewing the code, I've identified the following non-static functions as critical targets: * `trace_wprobe_create` (This is tricky since it's a static function in the `.c` file, but it's part of the `dyn_event_operations` structure. The compiler should not inline it, so it's a valid symbol.) * `wprobe_perf_handler` * `wprobe_trace_handler` * `wprobe_trigger` * `wprobe_trigger_cmd_parse` * `wprobe_work_func` * `modify_local_hw_breakpoint_addr` * `arch_modify_local_hw_breakpoint_addr` * `arch_install_hw_breakpoint` * `arch_uninstall_hw_breakpoint` * `hw_breakpoint_restore` I've ensured that these functions are specific to this feature and are not ubiquitous functions called by every program in the corpus. These functions all relate directly to the new wprobe feature and/or the hardware breakpoint manipulation logic, ensuring that fuzzing these functions directly tests the new functionality and associated changes. Finally, we need to determine the essential configuration options for enabling this functionality, `EnableConfigs`. The patch introduces these new features: * `WPROBE_EVENTS` * `WPROBE_TRIGGERS` * `PROBE_EVENTS_BTF_ARGS` * `TRACING` (dependency of WPROBE_EVENTS). `WPROBE_TRIGGERS` defaults to 'y' if `WPROBE_EVENTS` and `HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR` are enabled. Additionally, `PROBE_EVENTS_BTF_ARGS` is used for BTF support, guarded by a conditional, but important for full functionality. With this analysis, I've identified why the patch is worth fuzzing, specific functions to target, and necessary config options to enable the feature. Let's call `set-results`. |
| 10/2 | 2026/09/24 05:45 | tool | set-results |
0mArgs:{
"EnableConfigs": [
"WPROBE_EVENTS",
"WPROBE_TRIGGERS",
"PROBE_EVENTS_BTF_ARGS"
],
"FocusSymbols": [
"trace_wprobe_create",
"wprobe_perf_handler",
"wprobe_trace_handler",
"wprobe_trigger",
"wprobe_trigger_cmd_parse",
"wprobe_work_func",
"modify_local_hw_breakpoint_addr",
"arch_modify_local_hw_breakpoint_addr"
],
"Reasoning": "The patch introduces watchpoint probe (wprobe) dynamic event tracing and triggers (trace_wprobe.c), allowing userspace to configure hardware watchpoints on kernel memory addresses through the tracefs dynamic_events and trigger interfaces. It also adds new hardware breakpoint manipulation infrastructure (modify_local_hw_breakpoint_addr, arch_modify_local_hw_breakpoint_addr) and modifies x86 DR7 register synchronization across traps, NMIs, and MCEs. These changes introduce significant new parsing, concurrency, and hardware breakpoint handling logic directly exposed to userspace fuzzing.",
"WorthFuzzing": true
}Results: {
"EnableConfigs": [
"WPROBE_EVENTS",
"WPROBE_TRIGGERS",
"PROBE_EVENTS_BTF_ARGS"
],
"FocusSymbols": [
"trace_wprobe_create",
"wprobe_perf_handler",
"wprobe_trace_handler",
"wprobe_trigger",
"wprobe_trigger_cmd_parse",
"wprobe_work_func",
"modify_local_hw_breakpoint_addr",
"arch_modify_local_hw_breakpoint_addr"
],
"Reasoning": "The patch introduces watchpoint probe (wprobe) dynamic event tracing and triggers (trace_wprobe.c), allowing userspace to configure hardware watchpoints on kernel memory addresses through the tracefs dynamic_events and trigger interfaces. It also adds new hardware breakpoint manipulation infrastructure (modify_local_hw_breakpoint_addr, arch_modify_local_hw_breakpoint_addr) and modifies x86 DR7 register synchronization across traps, NMIs, and MCEs. These changes introduce significant new parsing, concurrency, and hardware breakpoint handling logic directly exposed to userspace fuzzing.",
"WorthFuzzing": true
} |
| 11/1 | 2026/09/24 05:45 | action | If |
1mArgs:{
"WorthFuzzing": true
}Results: null |
| 12/2 | 2026/09/24 05:45 | agent | kmsan-evaluator |
1mModel:core Args: null Results: {
"KMSANReasoning": "The patch series adds support for watchpoint probe (wprobe) dynamic tracing events and event triggers based on hardware breakpoints, along with x86 architecture support for modifying local hardware breakpoint addresses (using a per-CPU sequence counter `cpu_dr7_seq` to serialize `local_db_save`/`local_db_restore`).\n\nKMSAN vs. KASAN applicability analysis:\n1. Memory initialization: All heap structures (`trace_wprobe`, `wprobe_trigger_data`, `traceprobe_parse_context`) are allocated using zeroing allocators (`kzalloc_flex`, `kzalloc_obj`), and strings use `kstrdup`.\n2. Stack variables: In `arch/x86/include/asm/debugreg.h` and callers (`exc_machine_check_*`, `exc_nmi`, `exc_debug_kernel`), the newly introduced `dr7` and `dr7_seq` output pointers are written unconditionally on the first iteration of `local_db_save()` before any read or use in `local_db_restore()`. In `arch_modify_local_hw_breakpoint_addr()` and `kernel/trace/trace_wprobe.c`, all local variables are explicitly initialized before use.\n3. User-space data exposure / padding: No structures are sent to user space via `copy_to_user`, netlink, socket options, or ioctl outputs. Trace records written to the ftrace ring buffer use `struct wprobe_trace_entry_head`, which consists of `struct trace_entry` (8 bytes) and `unsigned long ip` (8 bytes), having no structure padding. Trace output is serialized as text through the standard tracing sequencer.\n4. Failure modes: Any potential bugs in this patch (such as concurrency issues across IRQ/NMI contexts, hardware breakpoint slot tracking, refcounting/lifetime of probe objects, or string parsing bounds) are memory corruption, concurrency, or control flow issues properly covered by KASAN, LOCKDEP, and standard kernel debugging tools.\n\nTherefore, the changes do not introduce or expose uninitialized memory risks, and a dedicated KMSAN session is not warranted.",
"NeedsKMSAN": false
}Instruction: You are an expert Linux kernel security engineer specializing in kernel memory
error detectors (KASAN and KMSAN). Your job is to review the provided patch series and
determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer)
fuzzing session in addition to standard KASAN fuzzing.
CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN:
- Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable
a comprehensive suite of debugging tools and sanitizers, including KASAN
(out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP
(locking bugs and deadlocks), UB-sanitizers, and memory corruption checks.
- KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap,
or page allocations) and kernel-to-user memory info-leaks.
Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN,
LOCKDEP, OR OTHER STANDARD BUG DETECTORS.
A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY
set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks
that are detected ONLY by KMSAN.
Look holistically at the patch series and surrounding code. Even if no direct
uninitialized field accesses or new buffer allocations are added in the diff itself,
a patch may alter control flow, bounds checking, or data length calculations in ways
that change how the rest of the code operates on existing buffers (e.g. allowing
uninitialized stack/heap memory to be read, copied to user space, or used in control
flow). Do not hesitate to use your code access tools to inspect the surrounding code,
called functions, and callers.
Set NeedsKMSAN=true ONLY IF the patch introduces or modifies:
1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb
attributes, ioctl output arguments, socket options, or BPF buffers) where fields
or structure padding might not be fully initialized/zeroed.
2. Conditional logic or branching that depends on potentially uninitialized variables
or struct fields.
3. Allocation or initialization of complex data structures where uninitialized fields
could be read later in reachable code paths.
4. Bounds checks, lengths, or logic in a way that allows surrounding code to access
uninitialized bytes of existing buffers.
Set NeedsKMSAN=false IF:
- The code changes primarily risk out-of-bounds access, array overflows, NULL pointer
dereferences, locking deadlocks, or use-after-free bugs (these are already caught
by KASAN, LOCKDEP, or standard bug detectors).
- All stack/heap structures touched or introduced by the patch are fully zeroed
or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied.
- The patch does not introduce any risk of uninitialized memory usage or info-leaks.
Use your code access tools to inspect the surrounding code if necessary, then provide
detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit aa59edb7da5ec405cf1653e89aee6e32a2266101
Author: syz-cluster <triage@syzkaller.com>
Date: Thu Sep 24 05:44:54 2026 +0000
syz-cluster: applied patch under review
diff --git a/Documentation/trace/index.rst b/Documentation/trace/index.rst
index f4058e8e92e3d..4f3fa60e4ecd9 100644
--- a/Documentation/trace/index.rst
+++ b/Documentation/trace/index.rst
@@ -36,6 +36,7 @@ the Linux kernel.
kprobes
kprobetrace
fprobetrace
+ wprobetrace
eprobetrace
fprobe
ring-buffer-design
diff --git a/Documentation/trace/wprobetrace.rst b/Documentation/trace/wprobetrace.rst
new file mode 100644
index 0000000000000..1d9b32d02d53b
--- /dev/null
+++ b/Documentation/trace/wprobetrace.rst
@@ -0,0 +1,200 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================================
+Watchpoint probe (wprobe) Event Tracing
+=======================================
+
+.. Author: Masami Hiramatsu <mhiramat@kernel.org>
+
+Overview
+--------
+
+Wprobe event is a dynamic event based on the hardware breakpoint, which is
+similar to other probe events, but it is for watching data access. It allows
+you to trace which code accesses a specified data.
+
+As same as other dynamic events, wprobe events are defined via
+`dynamic_events` interface file on tracefs.
+
+Synopsis of wprobe-events
+-------------------------
+::
+
+ w:[GRP/][EVENT] SPEC [FETCHARGS] : Probe on data access
+
+ GRP : Group name for wprobe. If omitted, use "wprobes" for it.
+ EVENT : Event name for wprobe. If omitted, an event name is
+ generated based on the address or symbol.
+ SPEC : Breakpoint specification.
+ [r|w|rw]@<ADDRESS|SYMBOL[+|-OFFS]>[:LENGTH]
+
+ r|w|rw : Access type, r for read, w for write, and rw for both.
+ Default is rw if omitted.
+ ADDRESS : Address to watch (hexadecimal). MUST be in kernel space.
+ SYMBOL[+|-OFFS] : Symbol name to watch. (Optional positive/negative offset)
+ LENGTH : Length of the data to watch in bytes. (1, 2, 4, or 8)
+ Default is 4.
+
+ FETCHARGS : Arguments. Each probe can have up to 128 args.
+ $addr : Fetch the accessing address.
+ $value : Fetch the memory value at the accessing address (same as +0($addr)).
+ @ADDR : Fetch memory at ADDR (ADDR should be in kernel)
+ @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol)
+ +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*1)(\*2)
+ \IMM : Store an immediate value to the argument.
+ NAME=FETCHARG : Set NAME as the argument name of FETCHARG.
+ FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types
+ (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types
+ (x8/x16/x32/x64), "char", "string", "ustring", "symbol", "symstr"
+ and bitfield are supported.
+ (STRUCT[,ASGN])FETCHARG->MEMBER[->MEMBER] : If BTF is supported, typecast
+ FETCHARG to a pointer to STRUCT and then dereference the
+ pointer defined by ->MEMBER. ASGN can be specified optionally.
+ If ASGN is specified, FETCHARG will be cast to the same offset
+ position as the ASGN member, rather than to the beginning of
+ the STRUCT.
+ (STRUCT[,ASGN])(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above
+ can also be used with another FETCHARG.
+
+ (\*1) This is useful for fetching a field of data structures.
+ (\*2) "u" means user-space dereference.
+
+For the details of TYPE, see :ref:`kprobetrace documentation <kprobetrace_types>`.
+
+Usage examples
+--------------
+Here is an example to add a wprobe event on a variable `jiffies`.
+::
+
+ # echo 'w:my_jiffies w@jiffies' >> dynamic_events
+ # cat dynamic_events
+ w:wprobes/my_jiffies w@jiffies
+ # echo 1 > events/wprobes/enable
+ # cat trace | head
+ # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
+ # | | | ||||| | |
+ <idle>-0 [000] d.Z1. 717.026259: my_jiffies: (tick_do_update_jiffies64+0xbe/0x130)
+ <idle>-0 [000] d.Z1. 717.026373: my_jiffies: (tick_do_update_jiffies64+0xbe/0x130)
+
+You can see the code which writes to `jiffies` is `tick_do_update_jiffies64()`.
+
+Notes
+-----
+Wprobe event does not disable itself even if the module is unloaded.
+For example, if you add a wprobe event on a module variable, and then
+unload the module, the wprobe event will still be enabled. This is for
+watching the address is used unexpectedly after the module is unloaded.
+
+Combination with trigger action
+-------------------------------
+The event trigger action can extend the utilization of this wprobe.
+
+- set_wprobe:WPEVENT:FIELD[+|-ADJUST][:COUNT]
+- set_wprobe:WPEVENT:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COUNT]
+- clear_wprobe:WPEVENT[:FIELD[+|-ADJUST][:COUNT]]
+- clear_wprobe:WPEVENT[:(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST][:COUNT]]
+
+Set these triggers to the target event, then the WPROBE event will be
+setup to trace the memory access at FIELD[+|-ADJUST] address.
+If BTF is supported, the target address can also be resolved using BTF
+struct offset resolution: `(STRUCT[,ASGN])EVENT_FIELD->MEMBER[+|-ADJUST]`.
+Here, EVENT_FIELD is an event field containing a pointer to STRUCT (or to
+the ASGN member if ASGN is specified), and MEMBER is the struct member whose
+offset is resolved automatically via BTF.
+When clear_wprobe is hit, if FIELD is NOT specified, the WPEVENT is
+forcibly cleared. If FIELD[+|-ADJUST] is set, it clears WPEVENT only
+if its watching address is the same as the FIELD[+|-ADJUST] value.
+If COUNT is specified, it will set/clear WPEVENT only if it hits COUNT
+times.
+
+Notes:
+- set_wprobe only works on the wprobe which is NOT set a valid address yet,
+ and it must be enabled after the trigger is set.
+- clear_wprobe only works on the wprobe which is set a valid address, and it
+ will be soft-disabled after the trigger is cleared.
+- Therefore, if a trigger sets/clears a wprobe, other/same trigger events
+ will not work (on the same event) while the wprobe is set.
+
+The set_wprobe trigger does not change the type and length, these
+must be set when creating a new wprobe.
+
+The WPROBE event must be disabled when setting the new trigger
+and it will be busy afterwards. Recommended usage is to add a new
+wprobe at invalid dummy address (-1) and keep disabled.
+
+Wprobe triggers only support target addresses in kernel memory. If a
+set_wprobe trigger evaluates to a user-space memory address or NULL
+pointer, the trigger action ignores the update and skips setting the
+watchpoint.
+
+Wprobe triggers are not supported on kprobe_events, because kprobes
+themselves can use software breakpoints which conflicts with wprobe
+operation.
+
+
+For example, trace the first 8 bytes of the dentry data structure passed
+to do_truncate() until it is deleted by dentry_kill().
+(Note: all tracefs setup uses '>>' so that it does not kick do_truncate())
+::
+
+ # echo 'w:watch rw@-1:8 address=$addr value=+0($addr)' >> dynamic_events
+ # echo 'f:truncate do_truncate dentry=$arg2' >> dynamic_events
+ # echo 'set_wprobe:watch:dentry' >> events/fprobes/truncate/trigger
+ # echo 'f:dentry_kill dentry_kill dentry=$arg1' >> dynamic_events
+ # echo 'clear_wprobe:watch:dentry' >> events/fprobes/dentry_kill/trigger
+ # echo 1 >> events/fprobes/truncate/enable
+ # echo 1 >> events/fprobes/dentry_kill/enable
+
+ # echo aaa > /tmp/hoge
+ # echo bbb > /tmp/hoge
+ # echo ccc > /tmp/hoge
+ # rm /tmp/hoge
+
+Then, the trace data will show::
+
+ # tracer: nop
+ #
+ # entries-in-buffer/entries-written: 32/32 #P:8
+ #
+ # _-----=> irqs-off/BH-disabled
+ # / _----=> need-resched
+ # | / _---=> hardirq/softirq
+ # || / _--=> preempt-depth
+ # ||| / _-=> migrate-disable
+ # |||| / delay
+ # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
+ # | | | ||||| | |
+ sh-107 [004] ...1. 9.990418: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+ sh-107 [004] ...1. 9.990914: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b3de78
+ sh-107 [004] ...1. 9.993175: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049ddd40
+ sh-107 [004] ..... 9.995198: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 9.995389: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db998
+ sh-107 [004] ..Zff 9.997503: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997509: watch: (path_openat+0x211/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997514: watch: (path_openat+0xa56/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..Zff 9.997518: watch: (path_openat+0xae2/0xda0) address=0xffff8880048083a8 value=0x8200080
+ sh-107 [004] ..... 9.997521: truncate: (do_truncate+0x4/0x120) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 9.997582: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004808270
+ sh-107 [004] ...1. 9.999365: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db728
+ sh-107 [004] ...1. 9.999388: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b1c000
+ rm-113 [005] ..Zff 10.000965: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000971: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000984: watch: (lookup_fast+0xaa/0x150) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.000988: watch: (path_lookupat+0x97/0x1e0) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001010: watch: (lookup_one_qstr_excl+0x28/0x140) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001014: watch: (lookup_one_qstr_excl+0xd1/0x140) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001018: watch: (may_delete_dentry+0x1c/0x200) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001021: watch: (may_delete_dentry+0x195/0x200) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] ..Zff 10.001031: watch: (vfs_unlink+0x5e/0x260) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] d.Z.. 10.001067: watch: (d_make_discardable+0x1b/0x40) address=0xffff8880048083a8 value=0x8200080
+ rm-113 [005] d.Z.. 10.001071: watch: (d_make_discardable+0x29/0x40) address=0xffff8880048083a8 value=0x200080
+ rm-113 [005] ...1. 10.001072: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+ rm-113 [005] ...1. 10.001218: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880048083a8
+ sh-107 [004] ...1. 10.001416: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db110
+ sh-107 [004] ...1. 10.001444: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff8880049db248
+ sh-107 [004] ...1. 10.001500: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+ sh-107 [004] ...1. 10.002067: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+ sh-107 [004] ...1. 10.904920: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004b41e78
+ sh-107 [004] ...1. 10.905129: dentry_kill: (dentry_kill+0x0/0x2c0) dentry=0xffff888004ad6618
+
+You can see the watch event is correctly configured on the dentry.
diff --git a/arch/Kconfig b/arch/Kconfig
index 45c6577723623..5292bd05d5d4d 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -474,6 +474,24 @@ config HAVE_MIXED_BREAKPOINTS_REGS
Select this option if your arch implements breakpoints under the
latter fashion.
+config HAVE_POST_BREAKPOINT_HOOK
+ bool
+ depends on HAVE_HW_BREAKPOINT
+ help
+ Depending on the arch implementation of hardware breakpoints,
+ some of them provide breakpoint hook after the target memory
+ is modified.
+ Select this option if your arch implements breakpoints overflow
+ handler hooks after the target memory is modified.
+
+config HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
+ bool
+ depends on HAVE_HW_BREAKPOINT
+ help
+ Select this if the architecture can modify the address of an
+ installed hardware breakpoint on the local CPU without releasing
+ and reserving its hardware slot.
+
config HAVE_USER_RETURN_NOTIFIER
bool
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 15fd9ec5ecacb..e4eaf0b607d30 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -249,6 +249,7 @@ config X86
select HAVE_FUNCTION_TRACER
select HAVE_GCC_PLUGINS
select HAVE_HW_BREAKPOINT
+ select HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
select HAVE_IOREMAP_PROT
select HAVE_IRQ_EXIT_ON_IRQ_STACK if X86_64
select HAVE_IRQ_TIME_ACCOUNTING
@@ -288,6 +289,7 @@ config X86
select MMU_GATHER_RCU_TABLE_FREE
select MMU_GATHER_MERGE_VMAS
select HAVE_POSIX_CPU_TIMERS_TASK_WORK
+ select HAVE_POST_BREAKPOINT_HOOK
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_RELIABLE_STACKTRACE if UNWINDER_ORC || STACK_VALIDATION
select HAVE_FUNCTION_ARG_ACCESS_API
diff --git a/arch/x86/include/asm/debugreg.h b/arch/x86/include/asm/debugreg.h
index 854d82b88ff4e..515d2d313d0c8 100644
--- a/arch/x86/include/asm/debugreg.h
+++ b/arch/x86/include/asm/debugreg.h
@@ -18,6 +18,7 @@
#define DR7_FIXED_1 0x00000400
DECLARE_PER_CPU(unsigned long, cpu_dr7);
+DECLARE_PER_CPU(unsigned int, cpu_dr7_seq);
#ifndef CONFIG_PARAVIRT_XXL
/*
@@ -125,40 +126,69 @@ static __always_inline bool hw_breakpoint_active(void)
extern void hw_breakpoint_restore(void);
-static __always_inline unsigned long local_db_save(void)
+static __always_inline void local_db_save(unsigned long *dr7,
+ unsigned int *dr7_seq)
{
- unsigned long dr7;
+ bool retrying = false;
- if (cpu_feature_enabled(X86_FEATURE_HYPERVISOR) && !hw_breakpoint_active())
- return 0;
+ do {
+ /* Inner loop: get DR7 with checking cpu_dr7_seq. */
+ *dr7_seq = this_cpu_read(cpu_dr7_seq);
- get_debugreg(dr7, 7);
+ if (unlikely(retrying)) {
+ *dr7 = this_cpu_read(cpu_dr7);
+ } else {
+ get_debugreg(*dr7, 7);
- /* Architecturally set bit */
- dr7 &= ~DR7_FIXED_1;
- if (dr7)
- set_debugreg(DR7_FIXED_1, 7);
+ /* Architecturally set bit */
+ *dr7 &= ~DR7_FIXED_1;
+ }
- /*
- * Ensure the compiler doesn't lower the above statements into
- * the critical section; disabling breakpoints late would not
- * be good.
- */
- barrier();
+ barrier();
+ if (unlikely(*dr7_seq != this_cpu_read(cpu_dr7_seq))) {
+ retrying = true;
+ continue;
+ }
+
+ /* Outer loop: clear DR7 and retry if sequence number is updated. */
+ if (*dr7)
+ set_debugreg(DR7_FIXED_1, 7);
- return dr7;
+ /*
+ * Ensure the compiler doesn't lower the above statements into
+ * the critical section; disabling breakpoints late would not
+ * be good.
+ */
+ barrier();
+ retrying = true;
+ } while (unlikely(*dr7_seq != this_cpu_read(cpu_dr7_seq)));
}
-static __always_inline void local_db_restore(unsigned long dr7)
+static __always_inline void local_db_restore(unsigned long dr7,
+ unsigned int dr7_seq)
{
+ unsigned long val;
+ unsigned int seq;
+
/*
* Ensure the compiler doesn't raise this statement into
* the critical section; enabling breakpoints early would
* not be good.
*/
barrier();
- if (dr7)
- set_debugreg(dr7, 7);
+ do {
+ seq = this_cpu_read(cpu_dr7_seq);
+ if (seq == dr7_seq) {
+ if (!dr7)
+ return;
+ val = dr7;
+ } else {
+ val = this_cpu_read(cpu_dr7);
+ }
+
+ set_debugreg(val | DR7_FIXED_1, 7);
+ barrier();
+ } while (unlikely(seq != this_cpu_read(cpu_dr7_seq)));
}
#ifdef CONFIG_CPU_SUP_AMD
diff --git a/arch/x86/include/asm/hw_breakpoint.h b/arch/x86/include/asm/hw_breakpoint.h
index 0bc931cd0698b..f35ec491f6dda 100644
--- a/arch/x86/include/asm/hw_breakpoint.h
+++ b/arch/x86/include/asm/hw_breakpoint.h
@@ -59,6 +59,8 @@ extern int hw_breakpoint_exceptions_notify(struct notifier_block *unused,
int arch_install_hw_breakpoint(struct perf_event *bp);
+int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr);
void arch_uninstall_hw_breakpoint(struct perf_event *bp);
void hw_breakpoint_pmu_read(struct perf_event *bp);
void hw_breakpoint_pmu_unthrottle(struct perf_event *bp);
diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
index ab469605fc893..b3669d6f88d4f 100644
--- a/arch/x86/kernel/cpu/mce/core.c
+++ b/arch/x86/kernel/cpu/mce/core.c
@@ -2108,6 +2108,10 @@ bool filter_mce(struct mce *m)
static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)
{
irqentry_state_t irq_state;
+ unsigned long dr7;
+ unsigned int dr7_seq;
+
+ local_db_save(&dr7, &dr7_seq);
WARN_ON_ONCE(user_mode(regs));
@@ -2116,20 +2120,27 @@ static __always_inline void exc_machine_check_kernel(struct pt_regs *regs)
* mce_check_crashing_cpu() for details.
*/
if (mca_cfg.initialized && mce_check_crashing_cpu())
- return;
+ goto out;
irq_state = irqentry_nmi_enter(regs);
do_machine_check(regs);
irqentry_nmi_exit(regs, irq_state);
+out:
+ local_db_restore(dr7, dr7_seq);
}
static __always_inline void exc_machine_check_user(struct pt_regs *regs)
{
+ unsigned long dr7;
+ unsigned int dr7_seq;
+
irqentry_enter_from_user_mode(regs);
+ local_db_save(&dr7, &dr7_seq);
do_machine_check(regs);
+ local_db_restore(dr7, dr7_seq);
irqentry_exit_to_user_mode(regs);
}
@@ -2138,21 +2149,13 @@ static __always_inline void exc_machine_check_user(struct pt_regs *regs)
/* MCE hit kernel mode */
DEFINE_IDTENTRY_MCE(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
exc_machine_check_kernel(regs);
- local_db_restore(dr7);
}
/* The user mode variant. */
DEFINE_IDTENTRY_MCE_USER(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
exc_machine_check_user(regs);
- local_db_restore(dr7);
}
#ifdef CONFIG_X86_FRED
@@ -2169,28 +2172,20 @@ DEFINE_IDTENTRY_MCE_USER(exc_machine_check)
*/
DEFINE_FREDENTRY_MCE(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
if (user_mode(regs))
exc_machine_check_user(regs);
else
exc_machine_check_kernel(regs);
- local_db_restore(dr7);
}
#endif
#else
/* 32bit unified entry point */
DEFINE_IDTENTRY_RAW(exc_machine_check)
{
- unsigned long dr7;
-
- dr7 = local_db_save();
if (user_mode(regs))
exc_machine_check_user(regs);
else
exc_machine_check_kernel(regs);
- local_db_restore(dr7);
}
#endif
diff --git a/arch/x86/kernel/hw_breakpoint.c b/arch/x86/kernel/hw_breakpoint.c
index f846c15f21ca1..b413adc2bbbf4 100644
--- a/arch/x86/kernel/hw_breakpoint.c
+++ b/arch/x86/kernel/hw_breakpoint.c
@@ -40,6 +40,8 @@
DEFINE_PER_CPU(unsigned long, cpu_dr7);
EXPORT_PER_CPU_SYMBOL(cpu_dr7);
+/* Sequence number of the per-CPU DR7 state. */
+DEFINE_PER_CPU(unsigned int, cpu_dr7_seq);
/* Per cpu debug address registers values */
static DEFINE_PER_CPU(unsigned long, cpu_debugreg[HBP_NUM]);
@@ -97,38 +99,95 @@ int decode_dr7(unsigned long dr7, int bpnum, unsigned *len, unsigned *type)
int arch_install_hw_breakpoint(struct perf_event *bp)
{
struct arch_hw_breakpoint *info = counter_arch_bp(bp);
- unsigned long *dr7;
+ unsigned int seq;
int i;
lockdep_assert_irqs_disabled();
- for (i = 0; i < HBP_NUM; i++) {
- struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
+ if (perf_guest_in_guest())
+ return -EBUSY;
- if (!*slot) {
- *slot = bp;
+ for (i = 0; i < HBP_NUM; i++) {
+ if (!this_cpu_cmpxchg(bp_per_reg[i], NULL, bp))
break;
- }
}
if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
return -EBUSY;
- set_debugreg(info->address, i);
- __this_cpu_write(cpu_debugreg[i], info->address);
+ do {
+ seq = this_cpu_inc_return(cpu_dr7_seq);
+ this_cpu_write(cpu_debugreg[i], info->address);
+ barrier();
+ set_debugreg(info->address, i);
+ if (info->mask)
+ amd_set_dr_addr_mask(info->mask, i);
+ this_cpu_or(cpu_dr7, encode_dr7(i, info->len, info->type));
+ barrier();
+ set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);
+ barrier();
+ } while (seq != this_cpu_read(cpu_dr7_seq));
- dr7 = this_cpu_ptr(&cpu_dr7);
- *dr7 |= encode_dr7(i, info->len, info->type);
+ return 0;
+}
- /*
- * Ensure we first write cpu_dr7 before we set the DR7 register.
- * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
- */
- barrier();
+/*
+ * Modify the address of an installed perf counter breakpoint on local CPU.
+ *
+ * This updates the debug address register and per-CPU shadow state without
+ * changing slot allocation or DR7. The core layer (modify_local_hw_breakpoint_addr())
+ * manages updating bp->attr.bp_addr and counter_arch_bp(bp)->address.
+ *
+ * Atomic: called with IRQs disabled and handles variables and registers
+ * local to this CPU.
+ */
+int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr)
+{
+ struct arch_hw_breakpoint *info = counter_arch_bp(bp);
+ struct arch_hw_breakpoint hw;
+ struct perf_event_attr attr = bp->attr;
+ unsigned int seq;
+ int i, ret;
+
+ lockdep_assert_irqs_disabled();
- set_debugreg(*dr7, 7);
- if (info->mask)
- amd_set_dr_addr_mask(info->mask, i);
+ if (perf_guest_in_guest())
+ return -EBUSY;
+
+ attr.bp_addr = addr;
+ ret = hw_breakpoint_arch_parse(bp, &attr, &hw);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < HBP_NUM; i++) {
+ if (this_cpu_read(bp_per_reg[i]) == bp)
+ break;
+ }
+
+ if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
+ return -ENOENT;
+
+ do {
+ seq = this_cpu_inc_return(cpu_dr7_seq);
+
+ /* Disable breakpoint in shadow DR7 and hardware DR7 */
+ this_cpu_and(cpu_dr7, ~__encode_dr7(i, info->len, info->type));
+ barrier();
+ set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);
+ barrier();
+
+ this_cpu_write(cpu_debugreg[i], addr);
+ barrier();
+ set_debugreg(addr, i);
+ barrier();
+
+ /* Re-enable breakpoint in shadow DR7 and hardware DR7 */
+ this_cpu_or(cpu_dr7, encode_dr7(i, info->len, info->type));
+ barrier();
+ set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);
+ barrier();
+ } while (seq != this_cpu_read(cpu_dr7_seq));
return 0;
}
@@ -146,36 +205,34 @@ void arch_uninstall_hw_breakpoint(struct perf_event *bp)
{
struct arch_hw_breakpoint *info = counter_arch_bp(bp);
unsigned long dr7;
+ unsigned int seq;
int i;
lockdep_assert_irqs_disabled();
for (i = 0; i < HBP_NUM; i++) {
- struct perf_event **slot = this_cpu_ptr(&bp_per_reg[i]);
-
- if (*slot == bp) {
- *slot = NULL;
+ if (this_cpu_read(bp_per_reg[i]) == bp)
break;
- }
}
if (WARN_ONCE(i == HBP_NUM, "Can't find any breakpoint slot"))
return;
- dr7 = this_cpu_read(cpu_dr7);
- dr7 &= ~__encode_dr7(i, info->len, info->type);
-
- set_debugreg(dr7, 7);
- if (info->mask)
- amd_set_dr_addr_mask(0, i);
-
- /*
- * Ensure the write to cpu_dr7 is after we've set the DR7 register.
- * This ensures an NMI never see cpu_dr7 0 when DR7 is not.
- */
- barrier();
-
- this_cpu_write(cpu_dr7, dr7);
+ do {
+ seq = this_cpu_inc_return(cpu_dr7_seq);
+ dr7 = this_cpu_read(cpu_dr7);
+ dr7 &= ~__encode_dr7(i, info->len, info->type);
+ set_debugreg(dr7 | DR7_FIXED_1, 7);
+ if (info->mask)
+ amd_set_dr_addr_mask(0, i);
+ barrier();
+ this_cpu_and(cpu_dr7,
+ ~__encode_dr7(i, info->len, info->type));
+ barrier();
+ } while (seq != this_cpu_read(cpu_dr7_seq));
+
+ WARN_ONCE(this_cpu_cmpxchg(bp_per_reg[i], bp, NULL) != bp,
+ "Can't release breakpoint slot");
}
static int arch_bp_generic_len(int x86_len)
@@ -309,13 +366,14 @@ static inline bool within_cpu_entry(unsigned long addr, unsigned long end)
sizeof(struct tlb_state)))
return true;
- /*
- * When in guest (X86_FEATURE_HYPERVISOR), local_db_save()
- * will read per-cpu cpu_dr7 before clear dr7 register.
- */
+ /* local_db_save() reads this state before clearing DR7. */
if (within_area(addr, end, (unsigned long)&per_cpu(cpu_dr7, cpu),
sizeof(cpu_dr7)))
return true;
+ if (within_area(addr, end,
+ (unsigned long)&per_cpu(cpu_dr7_seq, cpu),
+ sizeof(cpu_dr7_seq)))
+ return true;
}
return false;
@@ -483,12 +541,18 @@ void flush_ptrace_hw_breakpoint(struct task_struct *tsk)
void hw_breakpoint_restore(void)
{
- set_debugreg(__this_cpu_read(cpu_debugreg[0]), 0);
- set_debugreg(__this_cpu_read(cpu_debugreg[1]), 1);
- set_debugreg(__this_cpu_read(cpu_debugreg[2]), 2);
- set_debugreg(__this_cpu_read(cpu_debugreg[3]), 3);
- set_debugreg(DR6_RESERVED, 6);
- set_debugreg(__this_cpu_read(cpu_dr7), 7);
+ unsigned int seq;
+
+ do {
+ seq = this_cpu_inc_return(cpu_dr7_seq);
+ set_debugreg(this_cpu_read(cpu_debugreg[0]), 0);
+ set_debugreg(this_cpu_read(cpu_debugreg[1]), 1);
+ set_debugreg(this_cpu_read(cpu_debugreg[2]), 2);
+ set_debugreg(this_cpu_read(cpu_debugreg[3]), 3);
+ set_debugreg(DR6_RESERVED, 6);
+ set_debugreg(this_cpu_read(cpu_dr7) | DR7_FIXED_1, 7);
+ barrier();
+ } while (seq != this_cpu_read(cpu_dr7_seq));
}
EXPORT_SYMBOL_FOR_KVM(hw_breakpoint_restore);
diff --git a/arch/x86/kernel/nmi.c b/arch/x86/kernel/nmi.c
index 3c9f60d6ca5a7..c3806f7bd3e83 100644
--- a/arch/x86/kernel/nmi.c
+++ b/arch/x86/kernel/nmi.c
@@ -531,11 +531,12 @@ enum nmi_states {
};
static DEFINE_PER_CPU(enum nmi_states, nmi_state);
static DEFINE_PER_CPU(unsigned long, nmi_cr2);
-static DEFINE_PER_CPU(unsigned long, nmi_dr7);
DEFINE_IDTENTRY_RAW(exc_nmi)
{
irqentry_state_t irq_state;
+ unsigned long dr7;
+ unsigned int dr7_seq;
struct nmi_stats *nsp = this_cpu_ptr(&nmi_stats);
/*
@@ -572,7 +573,7 @@ DEFINE_IDTENTRY_RAW(exc_nmi)
*/
sev_es_ist_enter(regs);
- this_cpu_write(nmi_dr7, local_db_save());
+ local_db_save(&dr7, &dr7_seq);
irq_state = irqentry_nmi_enter(regs);
@@ -594,7 +595,7 @@ DEFINE_IDTENTRY_RAW(exc_nmi)
irqentry_nmi_exit(regs, irq_state);
- local_db_restore(this_cpu_read(nmi_dr7));
+ local_db_restore(dr7, dr7_seq);
sev_es_ist_exit();
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index 30aa8369957e9..018abe736285d 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -1231,8 +1231,12 @@ static noinstr void exc_debug_kernel(struct pt_regs *regs, unsigned long dr6)
* it results in an endless recursion and stack overflow. Thus we stay
* with the IDT approach, i.e., save DR7 and disable #DB.
*/
- unsigned long dr7 = local_db_save();
- irqentry_state_t irq_state = irqentry_nmi_enter(regs);
+ unsigned long dr7;
+ unsigned int dr7_seq;
+ irqentry_state_t irq_state;
+
+ local_db_save(&dr7, &dr7_seq);
+ irq_state = irqentry_nmi_enter(regs);
instrumentation_begin();
/*
@@ -1289,7 +1293,7 @@ static noinstr void exc_debug_kernel(struct pt_regs *regs, unsigned long dr6)
instrumentation_end();
irqentry_nmi_exit(regs, irq_state);
- local_db_restore(dr7);
+ local_db_restore(dr7, dr7_seq);
}
static noinstr void exc_debug_user(struct pt_regs *regs, unsigned long dr6)
diff --git a/include/linux/hw_breakpoint.h b/include/linux/hw_breakpoint.h
index db199d653dd1a..bf65c7fffd996 100644
--- a/include/linux/hw_breakpoint.h
+++ b/include/linux/hw_breakpoint.h
@@ -81,6 +81,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
void *context);
+int modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr);
+
extern int register_perf_hw_breakpoint(struct perf_event *bp);
extern void unregister_hw_breakpoint(struct perf_event *bp);
extern void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events);
@@ -124,6 +127,9 @@ register_wide_hw_breakpoint(struct perf_event_attr *attr,
perf_overflow_handler_t triggered,
void *context) { return NULL; }
static inline int
+modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr) { return -EOPNOTSUPP; }
+static inline int
register_perf_hw_breakpoint(struct perf_event *bp) { return -ENOSYS; }
static inline void unregister_hw_breakpoint(struct perf_event *bp) { }
static inline void
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 5842552294c19..194e69270460e 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -1677,8 +1677,8 @@ struct perf_guest_info_callbacks {
unsigned int (*state)(void);
unsigned long (*get_ip)(void);
unsigned int (*handle_intel_pt_intr)(void);
-
void (*handle_mediated_pmi)(void);
+ bool (*in_guest)(void);
};
#ifdef CONFIG_GUEST_PERF_EVENTS
@@ -1689,6 +1689,7 @@ DECLARE_STATIC_CALL(__perf_guest_state, *perf_guest_cbs->state);
DECLARE_STATIC_CALL(__perf_guest_get_ip, *perf_guest_cbs->get_ip);
DECLARE_STATIC_CALL(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->handle_intel_pt_intr);
DECLARE_STATIC_CALL(__perf_guest_handle_mediated_pmi, *perf_guest_cbs->handle_mediated_pmi);
+DECLARE_STATIC_CALL(__perf_guest_in_guest, *perf_guest_cbs->in_guest);
static inline unsigned int perf_guest_state(void)
{
@@ -1710,6 +1711,11 @@ static inline void perf_guest_handle_mediated_pmi(void)
static_call(__perf_guest_handle_mediated_pmi)();
}
+static inline bool perf_guest_in_guest(void)
+{
+ return static_call(__perf_guest_in_guest)();
+}
+
extern void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs);
@@ -1718,6 +1724,7 @@ extern void perf_unregister_guest_info_callbacks(struct perf_guest_info_callback
static inline unsigned int perf_guest_state(void) { return 0; }
static inline unsigned long perf_guest_get_ip(void) { return 0; }
static inline unsigned int perf_guest_handle_intel_pt_intr(void) { return 0; }
+static inline bool perf_guest_in_guest(void) { return false; }
#endif /* !CONFIG_GUEST_PERF_EVENTS */
diff --git a/include/linux/trace_events.h b/include/linux/trace_events.h
index 5cbd09c8be8d5..f81a5308c1167 100644
--- a/include/linux/trace_events.h
+++ b/include/linux/trace_events.h
@@ -337,6 +337,7 @@ enum {
TRACE_EVENT_FL_UPROBE_BIT,
TRACE_EVENT_FL_EPROBE_BIT,
TRACE_EVENT_FL_FPROBE_BIT,
+ TRACE_EVENT_FL_WPROBE_BIT,
TRACE_EVENT_FL_CUSTOM_BIT,
TRACE_EVENT_FL_TEST_STR_BIT,
};
@@ -367,6 +368,7 @@ enum {
TRACE_EVENT_FL_UPROBE = (1 << TRACE_EVENT_FL_UPROBE_BIT),
TRACE_EVENT_FL_EPROBE = (1 << TRACE_EVENT_FL_EPROBE_BIT),
TRACE_EVENT_FL_FPROBE = (1 << TRACE_EVENT_FL_FPROBE_BIT),
+ TRACE_EVENT_FL_WPROBE = (1 << TRACE_EVENT_FL_WPROBE_BIT),
TRACE_EVENT_FL_CUSTOM = (1 << TRACE_EVENT_FL_CUSTOM_BIT),
TRACE_EVENT_FL_TEST_STR = (1 << TRACE_EVENT_FL_TEST_STR_BIT),
};
@@ -736,6 +738,7 @@ enum event_trigger_type {
ETT_EVENT_HIST = (1 << 4),
ETT_HIST_ENABLE = (1 << 5),
ETT_EVENT_EPROBE = (1 << 6),
+ ETT_EVENT_WPROBE = (1 << 7),
};
extern int filter_match_preds(struct event_filter *filter, void *rec);
diff --git a/kernel/events/core.c b/kernel/events/core.c
index a6c8e38a31104..c5dead7c5f71d 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -7749,6 +7749,7 @@ DEFINE_STATIC_CALL_RET0(__perf_guest_state, *perf_guest_cbs->state);
DEFINE_STATIC_CALL_RET0(__perf_guest_get_ip, *perf_guest_cbs->get_ip);
DEFINE_STATIC_CALL_RET0(__perf_guest_handle_intel_pt_intr, *perf_guest_cbs->handle_intel_pt_intr);
DEFINE_STATIC_CALL_RET0(__perf_guest_handle_mediated_pmi, *perf_guest_cbs->handle_mediated_pmi);
+DEFINE_STATIC_CALL_RET0(__perf_guest_in_guest, *perf_guest_cbs->in_guest);
void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
{
@@ -7767,6 +7768,9 @@ void perf_register_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
if (cbs->handle_mediated_pmi)
static_call_update(__perf_guest_handle_mediated_pmi,
cbs->handle_mediated_pmi);
+
+ if (cbs->in_guest)
+ static_call_update(__perf_guest_in_guest, cbs->in_guest);
}
EXPORT_SYMBOL_GPL(perf_register_guest_info_callbacks);
@@ -7780,6 +7784,7 @@ void perf_unregister_guest_info_callbacks(struct perf_guest_info_callbacks *cbs)
static_call_update(__perf_guest_get_ip, (void *)&__static_call_return0);
static_call_update(__perf_guest_handle_intel_pt_intr, (void *)&__static_call_return0);
static_call_update(__perf_guest_handle_mediated_pmi, (void *)&__static_call_return0);
+ static_call_update(__perf_guest_in_guest, (void *)&__static_call_return0);
synchronize_rcu();
}
EXPORT_SYMBOL_GPL(perf_unregister_guest_info_callbacks);
diff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c
index 789add0c185a0..69bfb17b87c41 100644
--- a/kernel/events/hw_breakpoint.c
+++ b/kernel/events/hw_breakpoint.c
@@ -888,6 +888,54 @@ void unregister_wide_hw_breakpoint(struct perf_event * __percpu *cpu_events)
}
EXPORT_SYMBOL_GPL(unregister_wide_hw_breakpoint);
+/**
+ * modify_local_hw_breakpoint_addr - update a local breakpoint address
+ * @bp: the hwbp perf event for this CPU
+ * @addr: the new address for @bp
+ *
+ * Update only the address of an installed breakpoint on the local CPU without
+ * releasing and reserving its hardware slot. The caller must update other CPUs.
+ * Return 0, or -EOPNOTSUPP if the architecture does not support this operation.
+ *
+ * Note: bp->attr.bp_addr and counter_arch_bp(bp)->address are updated before
+ * modifying hardware registers so that interrupt/NMI handlers and CPU entry
+ * paths observe the new address during the transition. If an NMI hits the old
+ * breakpoint address before the hardware register update finishes, the event
+ * handler will see bp->attr.bp_addr already pointing to @addr. If the
+ * architecture update fails, both are rolled back to the previous address.
+ * Callers must be prepared for this behavior.
+ */
+#ifdef CONFIG_HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
+int modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr)
+{
+ unsigned long old_addr;
+ int ret;
+
+ lockdep_assert_irqs_disabled();
+
+ old_addr = bp->attr.bp_addr;
+ WRITE_ONCE(counter_arch_bp(bp)->address, addr);
+ WRITE_ONCE(bp->attr.bp_addr, addr);
+
+ ret = arch_modify_local_hw_breakpoint_addr(bp, addr);
+ if (ret) {
+ WRITE_ONCE(counter_arch_bp(bp)->address, old_addr);
+ WRITE_ONCE(bp->attr.bp_addr, old_addr);
+ return ret;
+ }
+
+ return 0;
+}
+#else
+int modify_local_hw_breakpoint_addr(struct perf_event *bp,
+ unsigned long addr)
+{
+ return -EOPNOTSUPP;
+}
+#endif
+EXPORT_SYMBOL_GPL(modify_local_hw_breakpoint_addr);
+
/**
* hw_breakpoint_is_used - check if breakpoints are currently used
*
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 0ab5916575a97..5fd8ed63c516d 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -862,6 +862,30 @@ config EPROBE_EVENTS
convert the type of an event field. For example, turn an
address into a string.
+config WPROBE_EVENTS
+ bool "Enable wprobe-based dynamic events"
+ depends on TRACING
+ depends on HAVE_HW_BREAKPOINT
+ depends on HAVE_POST_BREAKPOINT_HOOK
+ select PROBE_EVENTS
+ select DYNAMIC_EVENTS
+ help
+ This allows the user to add watchpoint tracing events based on
+ hardware breakpoints on the fly via the ftrace interface.
+
+ Those events can be inserted wherever hardware breakpoints can be
+ set, and record accessed memory address and values.
+
+config WPROBE_TRIGGERS
+ depends on WPROBE_EVENTS
+ depends on HAVE_MODIFY_LOCAL_HW_BREAKPOINT_ADDR
+ bool
+ default y
+ help
+ This adds an event trigger which will set the wprobe on a specific
+ field of an event. This allows user to trace the memory access of
+ an address pointed by the event field.
+
config BPF_EVENTS
depends on BPF_SYSCALL
depends on (KPROBE_EVENTS || UPROBE_EVENTS) && PERF_EVENTS
diff --git a/kernel/trace/Makefile b/kernel/trace/Makefile
index f934ff586bd43..141c8323de20b 100644
--- a/kernel/trace/Makefile
+++ b/kernel/trace/Makefile
@@ -126,6 +126,7 @@ obj-$(CONFIG_FTRACE_RECORD_RECURSION) += trace_recursion_record.o
obj-$(CONFIG_FPROBE) += fprobe.o
obj-$(CONFIG_RETHOOK) += rethook.o
obj-$(CONFIG_FPROBE_EVENTS) += trace_fprobe.o
+obj-$(CONFIG_WPROBE_EVENTS) += trace_wprobe.o
obj-$(CONFIG_TRACEPOINT_BENCHMARK) += trace_benchmark.o
obj-$(CONFIG_RV) += rv/
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c
index a946e0183fd1e..61d7d8bfb1577 100644
--- a/kernel/trace/trace.c
+++ b/kernel/trace/trace.c
@@ -4295,7 +4295,7 @@ static const char readme_msg[] =
"\t\t\t Write into this file to define/undefine new trace events.\n"
#endif
#if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS) || \
- defined(CONFIG_FPROBE_EVENTS)
+ defined(CONFIG_FPROBE_EVENTS) || defined(CONFIG_WPROBE_EVENTS)
"\t accepts: event-definitions (one definition per line)\n"
#if defined(CONFIG_KPROBE_EVENTS) || defined(CONFIG_UPROBE_EVENTS)
"\t Format: p[:[<group>/][<event>]] <place> [<args>]\n"
@@ -4305,6 +4305,9 @@ static const char readme_msg[] =
"\t f[:[<group>/][<event>]] <func-name>[%return] [<args>]\n"
"\t t[:[<group>/][<event>]] <tracepoint> [<args>]\n"
#endif
+#ifdef CONFIG_WPROBE_EVENTS
+ "\t w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>] [<args>]\n"
+#endif
#ifdef CONFIG_HIST_TRIGGERS
"\t s:[synthetic/]<event> <field> [<field>]\n"
#endif
@@ -4359,6 +4362,10 @@ static const char readme_msg[] =
"\t trigger: traceon, traceoff\n"
"\t enable_event:<system>:<event>\n"
"\t disable_event:<system>:<event>\n"
+#ifdef CONFIG_WPROBE_TRIGGERS
+ "\t set_wprobe:<wprobe-event>:<field>[+|-<offset>]\n"
+ "\t clear_wprobe:<wprobe-event>[:<field>[+|-<offset>]]\n"
+#endif
#ifdef CONFIG_HIST_TRIGGERS
"\t enable_hist:<system>:<event>\n"
"\t disable_hist:<system>:<event>\n"
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 74a7a50d1e78f..a61f840b18bdb 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -179,6 +179,11 @@ struct fexit_trace_entry_head {
unsigned long ret_ip;
};
+struct wprobe_trace_entry_head {
+ struct trace_entry ent;
+ unsigned long ip;
+};
+
#define TRACE_BUF_SIZE 1024
struct trace_array;
@@ -1978,6 +1983,7 @@ trigger_data_alloc(struct event_command *cmd_ops, char *cmd, char *param,
void *private_data);
extern void trigger_data_free(struct event_trigger_data *data);
extern int event_trigger_init(struct event_trigger_data *data);
+extern void event_trigger_free(struct event_trigger_data *data);
extern int trace_event_trigger_enable_disable(struct trace_event_file *file,
int trigger_enable);
extern void update_cond_flag(struct trace_event_file *file);
diff --git a/kernel/trace/trace_events_trigger.c b/kernel/trace/trace_events_trigger.c
index 149300cc5e8ad..120e6ec634b76 100644
--- a/kernel/trace/trace_events_trigger.c
+++ b/kernel/trace/trace_events_trigger.c
@@ -589,7 +589,7 @@ int event_trigger_init(struct event_trigger_data *data)
* Usually used directly as the @free method in event trigger
* implementations.
*/
-static void
+void
event_trigger_free(struct event_trigger_data *data)
{
if (WARN_ON_ONCE(data->ref <= 0))
diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c
index 804442b2f7d2b..ccf489635ad7f 100644
--- a/kernel/trace/trace_probe.c
+++ b/kernel/trace/trace_probe.c
@@ -20,7 +20,7 @@
#undef C
#define C(a, b) b
-static const char *trace_probe_err_text[] = { ERRORS };
+const char *trace_probe_err_text[] = { ERRORS };
static const char *reserved_field_names[] = {
"common_type",
@@ -890,6 +890,16 @@ static int query_btf_struct(const char *sname, struct traceprobe_parse_context *
ctx->struct_btf = NULL;
}
+ if (ctx->btf) {
+ id = btf_find_by_name_kind(ctx->btf, sname, BTF_KIND_STRUCT);
+ if (id > 0) {
+ btf_get(ctx->btf);
+ ctx->struct_btf = ctx->btf;
+ ctx->last_struct = btf_type_by_id(ctx->struct_btf, id);
+ return 0;
+ }
+ }
+
id = bpf_find_btf_id(sname, BTF_KIND_STRUCT, &btf);
if (id < 0)
return id;
@@ -965,7 +975,8 @@ static int handle_typecast(char *arg, struct traceprobe_parse_context *ctx)
if (!(tparg_is_event_probe(ctx->flags) ||
tparg_is_function_entry(ctx->flags) ||
- tparg_is_function_return(ctx->flags))) {
+ tparg_is_function_return(ctx->flags) ||
+ tparg_is_wprobe(ctx->flags))) {
trace_probe_log_err(ctx->offset, NOSUP_BTFARG);
return -EOPNOTSUPP;
}
@@ -1403,6 +1414,23 @@ static int parse_probe_vars(char *orig_arg, const struct fetch_type *t,
return 0;
}
+ /* wprobe only support "$addr" and "$value" variable */
+ if (ctx->flags & TPARG_FL_WPROBE) {
+ if (!strcmp(arg, "addr")) {
+ code->op = FETCH_OP_BADDR;
+ return 0;
+ }
+ if (!strcmp(arg, "value")) {
+ code->op = FETCH_OP_BADDR;
+ code++;
+ code->op = FETCH_OP_DEREF;
+ code->offset = 0;
+ *pcode = code;
+ return 0;
+ }
+ goto inval;
+ }
+
if (strcmp(arg, "comm") == 0 || strcmp(arg, "COMM") == 0) {
code->op = FETCH_OP_COMM;
return 0;
@@ -1462,8 +1490,8 @@ static int parse_probe_arg_register(char *arg, struct fetch_insn *code,
{
int ret;
- if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE)) {
- /* eprobe and fprobe do not handle registers */
+ if (ctx->flags & (TPARG_FL_TEVENT | TPARG_FL_FPROBE | TPARG_FL_WPROBE)) {
+ /* eprobe, fprobe and wprobe do not handle registers */
trace_probe_log_err(ctx->offset, BAD_VAR);
return -EINVAL;
}
diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h
index d1fb3520700fb..c54c554b1949d 100644
--- a/kernel/trace/trace_probe.h
+++ b/kernel/trace/trace_probe.h
@@ -90,6 +90,7 @@ typedef int (*print_type_func_t)(struct trace_seq *, void *, void *);
FETCH_OP(STACK, param), /* Stack: .param = index */ \
FETCH_OP(STACKP, none), /* Stack pointer */ \
FETCH_OP(RETVAL, none), /* Return value */ \
+ FETCH_OP(BADDR, none), /* Break address */ \
FETCH_OP(IMM, imm), /* Immediate: .immediate */ \
FETCH_OP(COMM, none), /* Current comm */ \
FETCH_OP(CURRENT, none), /* Current task_struct address */\
@@ -420,6 +421,7 @@ static inline int traceprobe_get_entry_data_size(struct trace_probe *tp)
#define TPARG_FL_USER BIT(4)
#define TPARG_FL_FPROBE BIT(5)
#define TPARG_FL_TPOINT BIT(6)
+#define TPARG_FL_WPROBE BIT(7)
#define TPARG_FL_LOC_MASK GENMASK(4, 0)
static inline bool tparg_is_function_entry(unsigned int flags)
@@ -437,6 +439,11 @@ static inline bool tparg_is_event_probe(unsigned int flags)
return !!(flags & TPARG_FL_TEVENT);
}
+static inline bool tparg_is_wprobe(unsigned int flags)
+{
+ return !!(flags & TPARG_FL_WPROBE);
+}
+
/* Each typecast consumes nested level. So the max number of typecast is 8. */
#define TRACEPROBE_MAX_NESTED_LEVEL 8
@@ -546,6 +553,10 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(ARG_TOO_LONG, "Argument expression is too long"), \
C(ARRAY_NO_CLOSE, "Array is not closed"), \
C(ARRAY_TOO_BIG, "Array number is too big"), \
+ C(BAD_ACCESS_ADDR, "Invalid access memory address"), \
+ C(BAD_ACCESS_FMT, "Access memory address requires @"), \
+ C(BAD_ACCESS_LEN, "This memory access length is not supported"), \
+ C(BAD_ACCESS_TYPE, "Bad memory access type"), \
C(BAD_ADDR_SUFFIX, "Invalid probed address suffix"), \
C(BAD_ARG_NAME, "Argument name must follow the same rules as C identifiers"), \
C(BAD_ARG_NUM, "Invalid argument number"), \
@@ -628,7 +639,14 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call,
C(TYPECAST_NOT_EVENT, "Typecasts are only for eprobe fields"), \
C(TYPECAST_REQ_FIELD, "Typecast requires a field access"), \
C(TYPECAST_SYM_OFFSET, "@SYM+/-OFFSET with typecast needs parentheses"), \
- C(USED_ARG_NAME, "This argument name is already used"),
+ C(USED_ARG_NAME, "This argument name is already used"), \
+ C(WPROBE_NO_MAXACT, "Watchpoint probe does not support maxactive"), \
+ C(WPROBE_NO_SIBLING, "Watchpoint probe does not support sibling probes"), \
+ C(WPROBE_ON_KPROBE, "Wprobe trigger is not supported on kprobe event"), \
+ C(WPROBE_NOT_FOUND, "Target wprobe event is not found"), \
+ C(WPROBE_BUSY, "Target wprobe event is already enabled"), \
+ C(WPROBE_NEED_FIELD, "Wprobe trigger requires a target field"), \
+ C(WPROBE_BAD_FIELD, "Target field must be pointer size"),
#undef C
#define C(a, b) TP_ERR_##a
@@ -652,6 +670,8 @@ void __trace_probe_log_err(int offset, int err);
DEFINE_FREE(trace_probe_log_clear, const char *, if (_T) trace_probe_log_clear())
+extern const char *trace_probe_err_text[];
+
#define trace_probe_log_err(offs, err) \
__trace_probe_log_err(offs, TP_ERR_##err)
diff --git a/kernel/trace/trace_wprobe.c b/kernel/trace/trace_wprobe.c
new file mode 100644
index 0000000000000..121310b72522f
--- /dev/null
+++ b/kernel/trace/trace_wprobe.c
@@ -0,0 +1,1668 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hardware-breakpoint-based tracing events
+ *
+ * Copyright (C) 2023, Masami Hiramatsu <mhiramat@kernel.org>
+ */
+#define pr_fmt(fmt) "trace_wprobe: " fmt
+
+#include <linux/atomic.h>
+#include <linux/compiler.h>
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/errno.h>
+#include <linux/hw_breakpoint.h>
+#include <linux/kallsyms.h>
+#include <linux/list.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/perf_event.h>
+#include <linux/rculist.h>
+#include <linux/security.h>
+#include <linux/spinlock.h>
+#include <linux/tracepoint.h>
+#include <linux/uaccess.h>
+#include <linux/workqueue.h>
+#include <linux/irq_work.h>
+#include <linux/preempt.h>
+
+#include <asm/ptrace.h>
+
+#include "trace.h"
+#include "trace_btf.h"
+#include "trace_dynevent.h"
+#include "trace_probe.h"
+#include "trace_probe_kernel.h"
+#include "trace_probe_tmpl.h"
+#include "trace_output.h"
+
+#define WPROBE_EVENT_SYSTEM "wprobes"
+
+static int trace_wprobe_create(const char *raw_command);
+static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev);
+static int trace_wprobe_release(struct dyn_event *ev);
+static bool trace_wprobe_is_busy(struct dyn_event *ev);
+static bool trace_wprobe_match(const char *system, const char *event,
+ int argc, const char **argv, struct dyn_event *ev);
+
+static struct dyn_event_operations trace_wprobe_ops = {
+ .create = trace_wprobe_create,
+ .show = trace_wprobe_show,
+ .is_busy = trace_wprobe_is_busy,
+ .free = trace_wprobe_release,
+ .match = trace_wprobe_match,
+};
+
+struct trace_wprobe {
+ struct dyn_event devent;
+ struct list_head active_list;
+ struct perf_event * __percpu *bp_event;
+ unsigned long addr;
+ int offset;
+ int len;
+ int type;
+ const char *symbol;
+ raw_spinlock_t lock;
+ struct irq_work irq_work;
+ struct work_struct work;
+ atomic_t missed;
+ /*
+ * work_pending is set to 1 before irq_work_queue() and cleared to 0
+ * after wprobe_work_func() finishes on_each_cpu(). This prevents a
+ * new trigger from overwriting tw->addr while the work is propagating
+ * the old address to per-CPU debug registers via IPI.
+ */
+ atomic_t work_pending;
+ struct trace_probe tp;
+};
+
+static bool is_trace_wprobe(struct dyn_event *ev)
+{
+ return ev->ops == &trace_wprobe_ops;
+}
+
+static struct trace_wprobe *to_trace_wprobe(struct dyn_event *ev)
+{
+ return container_of(ev, struct trace_wprobe, devent);
+}
+
+static DEFINE_MUTEX(wprobe_mutex);
+static LIST_HEAD(active_wprobes);
+
+#define for_each_trace_wprobe(pos, dpos) \
+ for_each_dyn_event(dpos) \
+ if (is_trace_wprobe(dpos) && (pos = to_trace_wprobe(dpos)))
+
+static bool trace_wprobe_is_busy(struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+
+ return trace_probe_is_enabled(&tw->tp);
+}
+
+static bool trace_wprobe_match(const char *system, const char *event,
+ int argc, const char **argv, struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+
+ if (event[0] != '\0' && strcmp(trace_probe_name(&tw->tp), event))
+ return false;
+
+ if (system && strcmp(trace_probe_group_name(&tw->tp), system))
+ return false;
+
+ return trace_probe_match_command_args(&tw->tp, argc, argv);
+}
+
+/*
+ * Note that we don't verify the fetch_insn code, since it does not come
+ * from user space.
+ */
+static int
+process_fetch_insn(struct fetch_insn *code, void *rec, void *edata,
+ void *dest, void *base)
+{
+ void *baddr = rec;
+ unsigned long val;
+ int ret;
+
+retry:
+ /* 1st stage: get value from context */
+ switch (code->op) {
+ case FETCH_OP_BADDR:
+ val = (unsigned long)baddr;
+ break;
+ case FETCH_NOP_SYMBOL: /* Ignore a place holder */
+ code++;
+ goto retry;
+ default:
+ ret = process_common_fetch_insn(code, &val);
+ if (ret < 0)
+ return ret;
+ }
+ code++;
+
+ return process_fetch_insn_bottom(code, val, dest, base);
+}
+NOKPROBE_SYMBOL(process_fetch_insn)
+
+static void wprobe_trace_handler(struct trace_wprobe *tw,
+ unsigned long addr,
+ struct pt_regs *regs,
+ struct trace_event_file *trace_file)
+{
+ struct wprobe_trace_entry_head *entry;
+ struct trace_event_call *call = trace_probe_event_call(&tw->tp);
+ struct trace_event_buffer fbuffer;
+ int dsize;
+
+ if (WARN_ON_ONCE(call != trace_file->event_call))
+ return;
+
+ if (trace_trigger_soft_disabled(trace_file))
+ return;
+
+ if (READ_ONCE(tw->addr) != addr)
+ return;
+
+ dsize = __get_data_size(&tw->tp, (void *)addr, NULL);
+
+ entry = trace_event_buffer_reserve(&fbuffer, trace_file,
+ sizeof(*entry) + tw->tp.size + dsize);
+ if (!entry)
+ return;
+
+ entry->ip = instruction_pointer(regs);
+ store_trace_args(&entry[1], &tw->tp, (void *)addr, NULL, sizeof(*entry), dsize);
+
+ fbuffer.regs = regs;
+ trace_event_buffer_commit(&fbuffer);
+}
+
+static void wprobe_perf_handler(struct perf_event *bp,
+ struct perf_sample_data *data,
+ struct pt_regs *regs)
+{
+ struct trace_wprobe *tw = bp->overflow_handler_context;
+ struct event_file_link *link;
+ unsigned long addr = bp->attr.bp_addr;
+
+ trace_probe_for_each_link_rcu(link, &tw->tp)
+ wprobe_trace_handler(tw, addr, regs, link->file);
+}
+
+static int __register_trace_wprobe(struct trace_wprobe *tw)
+{
+ struct perf_event_attr attr;
+ int i, ret;
+
+ if (tw->bp_event)
+ return -EINVAL;
+
+ for (i = 0; i < tw->tp.nr_args; i++) {
+ ret = traceprobe_update_arg(&tw->tp.args[i]);
+ if (ret)
+ return ret;
+ }
+
+ hw_breakpoint_init(&attr);
+ attr.bp_addr = tw->addr;
+ attr.bp_len = tw->len;
+ attr.bp_type = tw->type;
+
+ tw->bp_event = register_wide_hw_breakpoint(&attr, wprobe_perf_handler, tw);
+ if (IS_ERR_PCPU(tw->bp_event)) {
+ int ret = PTR_ERR_PCPU(tw->bp_event);
+
+ tw->bp_event = NULL;
+ return ret;
+ }
+
+ mutex_lock(&wprobe_mutex);
+ list_add(&tw->active_list, &active_wprobes);
+ mutex_unlock(&wprobe_mutex);
+
+ return 0;
+}
+
+static void __unregister_trace_wprobe(struct trace_wprobe *tw)
+{
+ if (tw->bp_event) {
+ mutex_lock(&wprobe_mutex);
+ list_del_init(&tw->active_list);
+ mutex_unlock(&wprobe_mutex);
+
+ irq_work_sync(&tw->irq_work);
+ cancel_work_sync(&tw->work);
+ unregister_wide_hw_breakpoint(tw->bp_event);
+ tw->bp_event = NULL;
+ }
+}
+
+static int wprobe_cpu_online(unsigned int cpu)
+{
+ struct trace_wprobe *tw;
+
+ mutex_lock(&wprobe_mutex);
+ list_for_each_entry(tw, &active_wprobes, active_list) {
+ if (tw->bp_event && !per_cpu(*tw->bp_event, cpu)) {
+ struct perf_event_attr attr;
+ struct perf_event *bp;
+
+ hw_breakpoint_init(&attr);
+ attr.bp_addr = READ_ONCE(tw->addr);
+ attr.bp_len = tw->len;
+ attr.bp_type = tw->type;
+
+ bp = perf_event_create_kernel_counter(&attr, cpu, NULL,
+ wprobe_perf_handler, tw);
+ if (!IS_ERR(bp))
+ per_cpu(*tw->bp_event, cpu) = bp;
+ }
+ }
+ mutex_unlock(&wprobe_mutex);
+
+ return 0;
+}
+
+static int wprobe_cpu_offline(unsigned int cpu)
+{
+ struct trace_wprobe *tw;
+
+ mutex_lock(&wprobe_mutex);
+ list_for_each_entry(tw, &active_wprobes, active_list) {
+ if (tw->bp_event) {
+ struct perf_event *bp = per_cpu(*tw->bp_event, cpu);
+
+ if (bp) {
+ per_cpu(*tw->bp_event, cpu) = NULL;
+ unregister_hw_breakpoint(bp);
+ }
+ }
+ }
+ mutex_unlock(&wprobe_mutex);
+
+ return 0;
+}
+
+static int trace_wprobe_update_local(struct trace_wprobe *tw, unsigned long addr)
+{
+ struct perf_event * __percpu *pevent;
+ struct perf_event *bp;
+
+ pevent = tw->bp_event;
+ if (!pevent)
+ return -EINVAL;
+
+ bp = *this_cpu_ptr(pevent);
+ if (!bp)
+ return -EINVAL;
+
+ if (bp->attr.bp_addr == addr)
+ return 0;
+
+ return modify_local_hw_breakpoint_addr(bp, addr);
+}
+
+static void wprobe_smp_update_func(void *info)
+{
+ struct trace_wprobe *tw = info;
+ unsigned long addr = READ_ONCE(tw->addr);
+
+ if (trace_wprobe_update_local(tw, addr))
+ atomic_inc(&tw->missed);
+}
+
+static void wprobe_work_func(struct work_struct *work)
+{
+ struct trace_wprobe *tw = container_of(work, struct trace_wprobe, work);
+
+ cpus_read_lock();
+ on_each_cpu(wprobe_smp_update_func, tw, true);
+ cpus_read_unlock();
+ /*
+ * Clear work_pending after all CPUs have updated their local debug
+ * registers. A new trigger may now update tw->addr and queue a new
+ * irq_work.
+ */
+ atomic_set(&tw->work_pending, 0);
+}
+
+static void wprobe_irq_work_func(struct irq_work *irq_work)
+{
+ struct trace_wprobe *tw = container_of(irq_work, struct trace_wprobe, irq_work);
+
+ schedule_work(&tw->work);
+}
+
+static void free_trace_wprobe(struct trace_wprobe *tw)
+{
+ if (tw) {
+ if (tw->work.func) {
+ irq_work_sync(&tw->irq_work);
+ cancel_work_sync(&tw->work);
+ }
+ trace_probe_cleanup(&tw->tp);
+ kfree(tw->symbol);
+ kfree(tw);
+ }
+}
+DEFINE_FREE(free_trace_wprobe, struct trace_wprobe *,
+ if (!IS_ERR_OR_NULL(_T))
+ free_trace_wprobe(_T))
+
+
+static struct trace_wprobe *alloc_trace_wprobe(const char *group,
+ const char *event,
+ const char *symbol,
+ int offset,
+ unsigned long addr,
+ int len, int type, int nargs)
+{
+ struct trace_wprobe *tw __free(free_trace_wprobe) = NULL;
+ int ret;
+
+ tw = kzalloc_flex(*tw, tp.args, nargs);
+ if (!tw)
+ return ERR_PTR(-ENOMEM);
+
+ raw_spin_lock_init(&tw->lock);
+ INIT_LIST_HEAD(&tw->active_list);
+ init_irq_work(&tw->irq_work, wprobe_irq_work_func);
+ INIT_WORK(&tw->work, wprobe_work_func);
+ atomic_set(&tw->missed, 0);
+ atomic_set(&tw->work_pending, 0);
+
+ if (symbol) {
+ tw->symbol = kstrdup(symbol, GFP_KERNEL);
+ if (!tw->symbol)
+ return ERR_PTR(-ENOMEM);
+ }
+ tw->offset = offset;
+ tw->addr = addr;
+ tw->len = len;
+ tw->type = type;
+
+ ret = trace_probe_init(&tw->tp, event, group, false, nargs);
+ if (ret < 0)
+ return ERR_PTR(ret);
+
+ dyn_event_init(&tw->devent, &trace_wprobe_ops);
+ return_ptr(tw);
+}
+
+static struct trace_wprobe *find_trace_wprobe(const char *event,
+ const char *group)
+{
+ struct dyn_event *pos;
+ struct trace_wprobe *tw;
+
+ for_each_trace_wprobe(tw, pos)
+ if (strcmp(trace_probe_name(&tw->tp), event) == 0 &&
+ strcmp(trace_probe_group_name(&tw->tp), group) == 0)
+ return tw;
+ return NULL;
+}
+
+static enum print_line_t
+print_wprobe_event(struct trace_iterator *iter, int flags,
+ struct trace_event *event)
+{
+ struct wprobe_trace_entry_head *field;
+ struct trace_seq *s = &iter->seq;
+ struct trace_probe *tp;
+
+ field = (struct wprobe_trace_entry_head *)iter->ent;
+ tp = trace_probe_primary_from_call(
+ container_of(event, struct trace_event_call, event));
+ if (WARN_ON_ONCE(!tp))
+ goto out;
+
+ trace_seq_printf(s, "%s: (", trace_probe_name(tp));
+
+ if (!seq_print_ip_sym_offset(s, field->ip, flags))
+ goto out;
+
+ trace_seq_putc(s, ')');
+
+ if (trace_probe_print_args(s, tp->args, tp->nr_args,
+ (u8 *)&field[1], field) < 0)
+ goto out;
+
+ trace_seq_putc(s, '\n');
+out:
+ return trace_handle_return(s);
+}
+
+static int wprobe_event_define_fields(struct trace_event_call *event_call)
+{
+ int ret;
+ struct wprobe_trace_entry_head field;
+ struct trace_probe *tp;
+
+ tp = trace_probe_primary_from_call(event_call);
+ if (WARN_ON_ONCE(!tp))
+ return -ENOENT;
+
+ DEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);
+
+ return traceprobe_define_arg_fields(event_call, sizeof(field), tp);
+}
+
+static struct trace_event_functions wprobe_funcs = {
+ .trace = print_wprobe_event
+};
+
+static struct trace_event_fields wprobe_fields_array[] = {
+ { .type = TRACE_FUNCTION_TYPE,
+ .define_fields = wprobe_event_define_fields },
+ {}
+};
+
+static int wprobe_register(struct trace_event_call *event,
+ enum trace_reg type, void *data);
+
+static inline void init_trace_event_call(struct trace_wprobe *tw)
+{
+ struct trace_event_call *call = trace_probe_event_call(&tw->tp);
+
+ call->event.funcs = &wprobe_funcs;
+ call->class->fields_array = wprobe_fields_array;
+ call->flags = TRACE_EVENT_FL_WPROBE;
+ call->class->reg = wprobe_register;
+}
+
+static int register_wprobe_event(struct trace_wprobe *tw)
+{
+ init_trace_event_call(tw);
+ return trace_probe_register_event_call(&tw->tp);
+}
+
+static int register_trace_wprobe_event(struct trace_wprobe *tw)
+{
+ struct trace_wprobe *old_tw;
+ int ret;
+
+ guard(mutex)(&event_mutex);
+
+ old_tw = find_trace_wprobe(trace_probe_name(&tw->tp),
+ trace_probe_group_name(&tw->tp));
+ if (old_tw) {
+ /*
+ * Wprobe does not support sibling probes because the event
+ * trigger (set_wprobe/clear_wprobe) identifies the target
+ * wprobe by its event name. Having multiple wprobes sharing
+ * the same event name would make the target ambiguous.
+ */
+ trace_probe_log_set_index(0);
+ trace_probe_log_err(0, WPROBE_NO_SIBLING);
+ return -EBUSY;
+ }
+
+ ret = register_wprobe_event(tw);
+ if (ret) {
+ trace_probe_log_set_index(0);
+ if (ret == -EEXIST)
+ trace_probe_log_err(0, EVENT_EXIST);
+ else if (ret != -ENOMEM)
+ trace_probe_log_err(0, FAIL_REG_PROBE);
+ return ret;
+ }
+
+ dyn_event_add(&tw->devent, trace_probe_event_call(&tw->tp));
+ return 0;
+}
+static int unregister_wprobe_event(struct trace_wprobe *tw)
+{
+ return trace_probe_unregister_event_call(&tw->tp);
+}
+
+static int unregister_trace_wprobe(struct trace_wprobe *tw)
+{
+ if (trace_probe_has_sibling(&tw->tp))
+ goto unreg;
+
+ if (trace_probe_is_enabled(&tw->tp))
+ return -EBUSY;
+
+ if (trace_event_dyn_busy(trace_probe_event_call(&tw->tp)))
+ return -EBUSY;
+
+ if (unregister_wprobe_event(tw))
+ return -EBUSY;
+
+unreg:
+ __unregister_trace_wprobe(tw);
+ dyn_event_remove(&tw->devent);
+ trace_probe_unlink(&tw->tp);
+
+ return 0;
+}
+
+static int enable_trace_wprobe(struct trace_event_call *call,
+ struct trace_event_file *file)
+{
+ struct trace_probe *tp;
+ struct trace_wprobe *tw;
+ bool enabled;
+ int ret = 0;
+
+ tp = trace_probe_primary_from_call(call);
+ if (WARN_ON_ONCE(!tp))
+ return -ENODEV;
+ enabled = trace_probe_is_enabled(tp);
+
+ if (file) {
+ ret = trace_probe_add_file(tp, file);
+ if (ret)
+ return ret;
+ } else {
+ trace_probe_set_flag(tp, TP_FLAG_PROFILE);
+ }
+
+ if (!enabled) {
+ list_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) {
+ ret = __register_trace_wprobe(tw);
+ if (ret < 0) {
+ struct trace_wprobe *tmp;
+
+ list_for_each_entry(tmp, trace_probe_probe_list(tp), tp.list) {
+ if (tmp == tw)
+ break;
+ __unregister_trace_wprobe(tmp);
+ }
+ if (file)
+ trace_probe_remove_file(tp, file);
+ else
+ trace_probe_clear_flag(tp, TP_FLAG_PROFILE);
+ return ret;
+ }
+ }
+ }
+
+ return 0;
+}
+
+static int disable_trace_wprobe(struct trace_event_call *call,
+ struct trace_event_file *file)
+{
+ struct trace_wprobe *tw;
+ struct trace_probe *tp;
+
+ tp = trace_probe_primary_from_call(call);
+ if (WARN_ON_ONCE(!tp))
+ return -ENODEV;
+
+ if (file) {
+ if (!trace_probe_get_file_link(tp, file))
+ return -ENOENT;
+ if (!trace_probe_has_single_file(tp))
+ goto out;
+ trace_probe_clear_flag(tp, TP_FLAG_TRACE);
+ } else {
+ trace_probe_clear_flag(tp, TP_FLAG_PROFILE);
+ }
+
+ if (!trace_probe_is_enabled(tp)) {
+ list_for_each_entry(tw, trace_probe_probe_list(tp), tp.list) {
+ __unregister_trace_wprobe(tw);
+ }
+ }
+
+out:
+ if (file)
+ trace_probe_remove_file(tp, file);
+
+ return 0;
+}
+
+static int wprobe_register(struct trace_event_call *event,
+ enum trace_reg type, void *data)
+{
+ struct trace_event_file *file = data;
+
+ switch (type) {
+ case TRACE_REG_REGISTER:
+ return enable_trace_wprobe(event, file);
+ case TRACE_REG_UNREGISTER:
+ return disable_trace_wprobe(event, file);
+
+#ifdef CONFIG_PERF_EVENTS
+ case TRACE_REG_PERF_REGISTER:
+ case TRACE_REG_PERF_UNREGISTER:
+ case TRACE_REG_PERF_OPEN:
+ case TRACE_REG_PERF_CLOSE:
+ case TRACE_REG_PERF_ADD:
+ case TRACE_REG_PERF_DEL:
+ return -EOPNOTSUPP;
+#endif
+ }
+ return 0;
+}
+
+static bool trace_wprobe_is_valid_addr(unsigned long addr, int len)
+{
+ if (addr < TASK_SIZE)
+ return false;
+
+ if (addr & (len - 1))
+ return false;
+
+ return true;
+}
+
+#ifdef CONFIG_WPROBE_TRIGGERS
+static u64 wprobe_trigger_clear_target __aligned(8);
+#define WPROBE_DEFAULT_CLEAR_ADDRESS ((unsigned long)&wprobe_trigger_clear_target)
+#endif
+
+static int parse_address_spec(const char *spec, unsigned long *addr, int *type,
+ int *len, char **symbol, int *offset)
+{
+ char *_spec __free(kfree) = NULL;
+ int _len = HW_BREAKPOINT_LEN_4;
+ int _type = HW_BREAKPOINT_RW;
+ unsigned long _addr = 0;
+ int _offset = 0;
+ char *at, *col;
+
+ _spec = kstrdup(spec, GFP_KERNEL);
+ if (!_spec)
+ return -ENOMEM;
+
+ at = strchr(_spec, '@');
+ col = strchr(_spec, ':');
+
+ if (!at) {
+ trace_probe_log_err(0, BAD_ACCESS_FMT);
+ return -EINVAL;
+ }
+
+ if (at != _spec) {
+ *at = '\0';
+
+ if (strcmp(_spec, "r") == 0)
+ _type = HW_BREAKPOINT_R;
+ else if (strcmp(_spec, "w") == 0)
+ _type = HW_BREAKPOINT_W;
+ else if (strcmp(_spec, "rw") == 0)
+ _type = HW_BREAKPOINT_RW;
+ else {
+ trace_probe_log_err(0, BAD_ACCESS_TYPE);
+ return -EINVAL;
+ }
+ }
+
+ if (col) {
+ *col = '\0';
+ if (kstrtoint(col + 1, 0, &_len)) {
+ trace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN);
+ return -EINVAL;
+ }
+
+ switch (_len) {
+ case 1:
+ _len = HW_BREAKPOINT_LEN_1;
+ break;
+ case 2:
+ _len = HW_BREAKPOINT_LEN_2;
+ break;
+ case 4:
+ _len = HW_BREAKPOINT_LEN_4;
+ break;
+ case 8:
+ _len = HW_BREAKPOINT_LEN_8;
+ break;
+ default:
+ trace_probe_log_err(col + 1 - _spec, BAD_ACCESS_LEN);
+ return -EINVAL;
+ }
+ }
+
+#ifdef CONFIG_WPROBE_TRIGGERS
+ if (strcmp(at + 1, "-1") == 0) {
+ _addr = WPROBE_DEFAULT_CLEAR_ADDRESS;
+ } else if (kstrtoul(at + 1, 0, &_addr) != 0) {
+#else
+ if (kstrtoul(at + 1, 0, &_addr) != 0) {
+#endif
+ char *off_str = strpbrk(at + 1, "+-");
+
+ if (off_str) {
+ if (kstrtoint(off_str, 0, &_offset) != 0) {
+ trace_probe_log_err(off_str - _spec, BAD_PROBE_ADDR);
+ return -EINVAL;
+ }
+ *off_str = '\0';
+ }
+ _addr = kallsyms_lookup_name(at + 1);
+ if (!_addr) {
+ trace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR);
+ return -ENOENT;
+ }
+ _addr += _offset;
+ *symbol = kstrdup(at + 1, GFP_KERNEL);
+ if (!*symbol)
+ return -ENOMEM;
+ }
+
+ if (!trace_wprobe_is_valid_addr(_addr, _len)) {
+ trace_probe_log_err(at + 1 - _spec, BAD_ACCESS_ADDR);
+ return -EINVAL;
+ }
+
+ *addr = _addr;
+ *offset = _offset;
+ *type = _type;
+ *len = _len;
+ return 0;
+}
+
+static int __trace_wprobe_create(int argc, const char *argv[])
+{
+ /*
+ * Argument syntax:
+ * w[:[GRP/][EVENT]] SPEC
+ *
+ * SPEC:
+ * [r|w|rw]@[ADDR|SYMBOL[+OFFS]][:LEN]
+ */
+ struct traceprobe_parse_context *ctx __free(traceprobe_parse_context) = NULL;
+ struct trace_wprobe *tw __free(free_trace_wprobe) = NULL;
+ const char *event = NULL, *group = WPROBE_EVENT_SYSTEM;
+ const char *tplog __free(trace_probe_log_clear) = NULL;
+ char *symbol __free(kfree) = NULL;
+ char *gbuf __free(kfree) = NULL;
+ char *ebuf __free(kfree) = NULL;
+ unsigned long addr;
+ int len, type, offset, i;
+ int ret;
+
+ if (argv[0][0] != 'w')
+ return -ECANCELED;
+
+ tplog = trace_probe_log_init("wprobe", argc, argv);
+
+ if (argc < 2) {
+ trace_probe_log_set_index(0);
+ trace_probe_log_err(0, NO_ARG_BODY);
+ return -EINVAL;
+ }
+
+ if (argv[0][1] != '\0') {
+ if (argv[0][1] != ':') {
+ trace_probe_log_set_index(0);
+ trace_probe_log_err(1, WPROBE_NO_MAXACT);
+ return -EINVAL;
+ }
+ event = &argv[0][2];
+ }
+
+ trace_probe_log_set_index(1);
+ ret = parse_address_spec(argv[1], &addr, &type, &len, &symbol, &offset);
+ if (ret < 0)
+ return ret;
+
+ trace_probe_log_set_index(0);
+ if (event) {
+ gbuf = kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL);
+ if (!gbuf)
+ return -ENOMEM;
+ ret = traceprobe_parse_event_name(&event, &group, gbuf,
+ event - argv[0]);
+ if (ret)
+ return ret;
+ }
+
+ if (!event) {
+ /* Make a new event name */
+ ebuf = kmalloc(MAX_EVENT_NAME_LEN, GFP_KERNEL);
+ if (!ebuf)
+ return -ENOMEM;
+ if (symbol)
+ snprintf(ebuf, MAX_EVENT_NAME_LEN, "%s", symbol);
+ else
+ snprintf(ebuf, MAX_EVENT_NAME_LEN, "w_0x%lx", addr);
+ sanitize_event_name(ebuf);
+ event = ebuf;
+ }
+
+ argc -= 2; argv += 2;
+ if (argc > MAX_TRACE_ARGS) {
+ trace_probe_log_set_index(2);
+ trace_probe_log_err(0, TOO_MANY_ARGS);
+ return -E2BIG;
+ }
+ tw = alloc_trace_wprobe(group, event, symbol, offset, addr, len, type, argc);
+ if (IS_ERR(tw))
+ return PTR_ERR(tw);
+
+ ctx = kzalloc_obj(*ctx);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->flags = TPARG_FL_KERNEL | TPARG_FL_WPROBE;
+
+ /* parse arguments */
+ for (i = 0; i < argc; i++) {
+ trace_probe_log_set_index(i + 2);
+ ctx->offset = 0;
+ ret = traceprobe_parse_probe_arg(&tw->tp, i, argv[i], ctx);
+ if (ret)
+ return ret; /* This can be -ENOMEM */
+ }
+
+ ret = traceprobe_set_print_fmt(&tw->tp, PROBE_PRINT_NORMAL);
+ if (ret < 0)
+ return ret;
+
+ ret = register_trace_wprobe_event(tw);
+ if (!ret)
+ tw = NULL; /* To avoid free */
+
+ return ret;
+}
+
+static int trace_wprobe_create(const char *raw_command)
+{
+ return trace_probe_create(raw_command, __trace_wprobe_create);
+}
+
+static int trace_wprobe_release(struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+ int ret = unregister_trace_wprobe(tw);
+
+ if (!ret)
+ free_trace_wprobe(tw);
+ return ret;
+}
+
+static int trace_wprobe_show(struct seq_file *m, struct dyn_event *ev)
+{
+ struct trace_wprobe *tw = to_trace_wprobe(ev);
+ const char *type_str;
+ int i, len;
+
+ seq_printf(m, "w:%s/%s", trace_probe_group_name(&tw->tp),
+ trace_probe_name(&tw->tp));
+
+ if (tw->type == HW_BREAKPOINT_R)
+ type_str = "r";
+ else if (tw->type == HW_BREAKPOINT_W)
+ type_str = "w";
+ else
+ type_str = "rw";
+
+ if (tw->len == HW_BREAKPOINT_LEN_1)
+ len = 1;
+ else if (tw->len == HW_BREAKPOINT_LEN_2)
+ len = 2;
+ else if (tw->len == HW_BREAKPOINT_LEN_4)
+ len = 4;
+ else
+ len = 8;
+
+ if (tw->symbol) {
+ if (tw->offset)
+ seq_printf(m, " %s@%s%+d:%d", type_str, tw->symbol,
+ tw->offset, len);
+ else
+ seq_printf(m, " %s@%s:%d", type_str, tw->symbol, len);
+#ifdef CONFIG_WPROBE_TRIGGERS
+ } else if (tw->addr == WPROBE_DEFAULT_CLEAR_ADDRESS) {
+ seq_printf(m, " %s@-1:%d", type_str, len);
+#endif
+ } else {
+ seq_printf(m, " %s@0x%lx:%d", type_str, tw->addr, len);
+ }
+
+ for (i = 0; i < tw->tp.nr_args; i++)
+ seq_printf(m, " %s=%s", tw->tp.args[i].name, tw->tp.args[i].comm);
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static enum cpuhp_state wprobe_hp_state __ro_after_init;
+
+static __init int init_wprobe_trace(void)
+{
+ int ret;
+
+ ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "trace/wprobe:online",
+ wprobe_cpu_online, wprobe_cpu_offline);
+ if (ret < 0)
+ return ret;
+
+ wprobe_hp_state = ret;
+
+ ret = dyn_event_register(&trace_wprobe_ops);
+ if (ret)
+ cpuhp_remove_state(wprobe_hp_state);
+
+ return ret;
+}
+fs_initcall(init_wprobe_trace);
+
+#ifdef CONFIG_WPROBE_TRIGGERS
+
+#define SET_WPROBE_STR "set_wprobe"
+#define CLEAR_WPROBE_STR "clear_wprobe"
+#define wprobe_trigger_log_err(file, glob, offs, err) \
+ tracing_log_err((file)->tr, "wprobe_trigger", glob, \
+ trace_probe_err_text, TP_ERR_##err, offs)
+
+struct wprobe_trigger_data {
+ struct rcu_head rcu;
+ struct trace_event_file *file;
+ struct trace_wprobe *tw;
+ int offset;
+ long adjust;
+ const char *field;
+ bool clear;
+};
+
+static bool __wprobe_count_func(struct event_trigger_data *data,
+ unsigned long addr)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long cur_addr;
+
+ if (in_nmi())
+ return false;
+
+ if (atomic_read(&tw->work_pending))
+ return false;
+
+ cur_addr = READ_ONCE(tw->addr);
+
+ if (!wprobe_data->clear) {
+ if (cur_addr != WPROBE_DEFAULT_CLEAR_ADDRESS)
+ return false;
+ if (!trace_wprobe_is_valid_addr(addr, tw->len))
+ return false;
+ } else {
+ if (cur_addr == WPROBE_DEFAULT_CLEAR_ADDRESS)
+ return false;
+ if (wprobe_data->field && cur_addr != addr)
+ return false;
+ }
+
+ return true;
+}
+
+static bool wprobe_count_func(struct event_trigger_data *data,
+ struct trace_buffer *buffer, void *rec,
+ struct ring_buffer_event *event)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long addr = 0;
+
+ if (!data->count)
+ return false;
+
+ if (wprobe_data->field) {
+ if (!rec)
+ return false;
+ addr = *(unsigned long *)((char *)rec + wprobe_data->offset);
+ addr += wprobe_data->adjust;
+ }
+
+ if (!__wprobe_count_func(data, addr)) {
+ atomic_inc(&tw->missed);
+ return false;
+ }
+
+ if (data->count != -1)
+ (data->count)--;
+
+ return true;
+}
+
+static void wprobe_trigger(struct event_trigger_data *data,
+ struct trace_buffer *buffer, void *rec,
+ struct ring_buffer_event *event)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ struct trace_wprobe *tw = wprobe_data->tw;
+ unsigned long target_addr, addr = 0, flags;
+
+ if (wprobe_data->field) {
+ if (!rec)
+ return;
+ addr = *(unsigned long *)((char *)rec + wprobe_data->offset);
+ addr += wprobe_data->adjust;
+ }
+
+ raw_spin_lock_irqsave(&tw->lock, flags);
+
+ if (!(data->flags & EVENT_TRIGGER_FL_COUNT)) {
+ if (!__wprobe_count_func(data, addr)) {
+ atomic_inc(&tw->missed);
+ goto out;
+ }
+ }
+
+ target_addr = wprobe_data->clear ? WPROBE_DEFAULT_CLEAR_ADDRESS : addr;
+
+ /*
+ * Try updating the hardware breakpoint on the local CPU first.
+ * If this fails, skip updating tw->addr and do not queue irq_work.
+ * Note that this wprobe will not be traced until tw_addr is updated.
+ * (e.g. NMI happens on this CPU.)
+ */
+ if (trace_wprobe_update_local(tw, target_addr)) {
+ atomic_inc(&tw->missed);
+ goto out;
+ }
+
+ WRITE_ONCE(tw->addr, target_addr);
+ if (!wprobe_data->clear)
+ clear_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+ else
+ set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+
+ /*
+ * For other CPUs, kick irq_work to asynchronously propagate
+ * the new address.
+ */
+ if (num_online_cpus() > 1) {
+ atomic_set(&tw->work_pending, 1);
+ irq_work_queue(&tw->irq_work);
+ }
+
+out:
+ raw_spin_unlock_irqrestore(&tw->lock, flags);
+}
+
+static void free_wprobe_trigger_data(struct wprobe_trigger_data *wprobe_data)
+{
+ if (wprobe_data) {
+ kfree(wprobe_data->field);
+ kfree(wprobe_data);
+ }
+}
+DEFINE_FREE(free_wprobe_trigger_data, struct wprobe_trigger_data *, free_wprobe_trigger_data(_T));
+
+static void free_private_wprobe_trigger_data(struct event_trigger_data *data)
+{
+ free_wprobe_trigger_data(data->private_data);
+}
+
+static int wprobe_trigger_print(struct seq_file *m,
+ struct event_trigger_data *data)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+ int missed = atomic_read(&wprobe_data->tw->missed);
+
+ if (wprobe_data->clear) {
+ seq_printf(m, "%s:%s", CLEAR_WPROBE_STR,
+ trace_event_name(wprobe_data->file->event_call));
+ if (wprobe_data->field) {
+ seq_printf(m, ":%s%+ld",
+ wprobe_data->field, wprobe_data->adjust);
+ }
+ } else {
+ seq_printf(m, "%s:%s:%s%+ld", SET_WPROBE_STR,
+ trace_event_name(wprobe_data->file->event_call),
+ wprobe_data->field, wprobe_data->adjust);
+ }
+
+ if (data->count == -1)
+ seq_puts(m, ":unlimited");
+ else
+ seq_printf(m, ":count=%ld", data->count);
+
+ if (data->filter_str)
+ seq_printf(m, " if %s", data->filter_str);
+
+ if (missed)
+ seq_printf(m, " # missed: %d", missed);
+
+ seq_putc(m, '\n');
+
+ return 0;
+}
+
+static struct wprobe_trigger_data *
+wprobe_trigger_alloc(struct trace_wprobe *tw, struct trace_event_file *file,
+ bool clear)
+{
+ struct wprobe_trigger_data *wprobe_data;
+
+ wprobe_data = kzalloc_obj(*wprobe_data);
+ if (!wprobe_data)
+ return NULL;
+
+ wprobe_data->tw = tw;
+ wprobe_data->clear = clear;
+ wprobe_data->file = file;
+
+ return wprobe_data;
+}
+
+static void wprobe_trigger_free(struct event_trigger_data *data)
+{
+ struct wprobe_trigger_data *wprobe_data = data->private_data;
+
+ if (WARN_ON_ONCE(data->ref <= 0))
+ return;
+
+ data->ref--;
+ if (!data->ref) {
+ /*
+ * Ensure SOFT_DISABLED flag is restored before clearing soft
+ * mode so that if the user did not explicitly enable this
+ * event, __ftrace_event_enable_disable() unregisters it.
+ */
+ set_bit(EVENT_FILE_FL_SOFT_DISABLED_BIT, &wprobe_data->file->flags);
+ /* Remove the SOFT_MODE flag */
+ trace_event_enable_disable(wprobe_data->file, 0, 1);
+ trace_event_put_ref(wprobe_data->file->event_call);
+ trigger_data_free(data);
+ }
+}
+
+#ifdef CONFIG_PROBE_EVENTS_BTF_ARGS
+
+static int get_offset_of_field(struct btf *btf, const struct btf_type *type, char *field_name)
+{
+ const struct btf_member *field;
+ const struct btf_type *mtype;
+ int bitoffs = 0;
+ u32 anon_offs;
+ char *next;
+
+ do {
+ next = strchr(field_name, '.');
+ if (next)
+ *next++ = '\0';
+
+ field = btf_find_struct_member(btf, type, field_name, &anon_offs, &mtype);
+ if (IS_ERR_OR_NULL(field))
+ return -ENOENT;
+
+ if (btf_type_kflag(mtype)) {
+ /* Reject bitfield member access */
+ if (BTF_MEMBER_BITFIELD_SIZE(field->offset))
+ return -EINVAL;
+ bitoffs += anon_offs + BTF_MEMBER_BIT_OFFSET(field->offset);
+ } else {
+ bitoffs += anon_offs + field->offset;
+ }
+
+ field_name = next;
+ if (next) {
+ type = btf_type_skip_modifiers(btf, field->type, NULL);
+ if (!type)
+ return -ENOENT;
+ }
+ } while (next);
+ return bitoffs / BITS_PER_BYTE;
+}
+
+/* btf_put(NULL) is acceptable. */
+DEFINE_FREE(btf_put, struct btf *, btf_put(_T))
+
+/* parse typecast: (TYPE[,ASGN])EVENT_FIELD->FIELD[.SUBFIELD...][+-OFFS] and set adjust. */
+static int wprobe_trigger_typecast_parse(char *field_str,
+ struct trace_event_file *file,
+ struct wprobe_trigger_data *wprobe_data,
+ const char *glob)
+{
+ struct btf *btf __free(btf_put) = NULL;
+ char *buf __free(kfree) = NULL;
+ struct ftrace_event_field *field;
+ const struct btf_type *type;
+ char *assign_field;
+ char *event_field;
+ char *type_field;
+ char *type_name;
+ char *offs;
+ long val = 0;
+ int base_offset = field_str - glob;
+ int event_field_offset;
+ int id, adjust;
+
+ buf = kstrdup(field_str, GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ type_name = buf + 1;
+ event_field = strchr(type_name, ')');
+ if (!event_field) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_name - buf),
+ DEREF_OPEN_BRACE);
+ return -EINVAL;
+ }
+ *event_field++ = '\0';
+
+ /* Check the optional assign field. */
+ assign_field = strchr(type_name, ',');
+ if (assign_field)
+ *assign_field++ = '\0';
+
+ /* Get the type field name. */
+ type_field = strstr(event_field, "->");
+ if (!type_field) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (event_field - buf),
+ TYPECAST_REQ_FIELD);
+ return -EINVAL;
+ }
+ *type_field = '\0';
+ type_field += 2;
+
+ offs = strpbrk(type_field, "+-");
+ if (offs) {
+ if (kstrtol(offs, 0, &val) < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (offs - buf),
+ BAD_DEREF_OFFS);
+ return -EINVAL;
+ }
+ *offs = '\0';
+ }
+
+ /* find type from BTF */
+ id = bpf_find_btf_id(type_name, BTF_KIND_STRUCT, &btf);
+ if (id < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_name - buf),
+ BAD_BTF_TID);
+ return id;
+ }
+
+ type = btf_type_by_id(btf, id);
+ if (!type) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_name - buf),
+ BAD_BTF_TID);
+ return -EINVAL;
+ }
+
+ adjust = get_offset_of_field(btf, type, type_field);
+ if (adjust < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (type_field - buf),
+ NO_BTF_FIELD);
+ return adjust;
+ }
+ wprobe_data->adjust = adjust + val;
+
+ if (assign_field) {
+ /* assign_field should be a struct field */
+ adjust = get_offset_of_field(btf, type, assign_field);
+ if (adjust < 0) {
+ wprobe_trigger_log_err(file, glob,
+ base_offset + (assign_field - buf),
+ NO_BTF_FIELD);
+ return adjust;
+ }
+ wprobe_data->adjust -= adjust;
+ }
+
+ event_field_offset = base_offset + (event_field - buf);
+ field = trace_find_event_field(file->event_call, event_field);
+ if (!field) {
+ wprobe_trigger_log_err(file, glob, event_field_offset, NO_EVENT_FIELD);
+ return -ENOENT;
+ }
+ if (field->size != sizeof(void *)) {
+ wprobe_trigger_log_err(file, glob, event_field_offset, WPROBE_BAD_FIELD);
+ return -ENOEXEC;
+ }
+ wprobe_data->offset = field->offset;
+ wprobe_data->field = kstrdup(event_field, GFP_KERNEL);
+ if (!wprobe_data->field)
+ return -ENOMEM;
+
+ return 0;
+}
+#else
+static int wprobe_trigger_typecast_parse(char *field_str,
+ struct trace_event_file *file,
+ struct wprobe_trigger_data *wprobe_data,
+ const char *glob)
+{
+ wprobe_trigger_log_err(file, glob, field_str - glob, NOSUP_BTFARG);
+ return -EOPNOTSUPP;
+}
+#endif /* CONFIG_PROBE_EVENTS_BTF_ARGS */
+
+static int wprobe_trigger_field_parse(char *field_str, struct trace_event_file *file,
+ struct wprobe_trigger_data *wprobe_data,
+ const char *glob)
+{
+ struct ftrace_event_field *field;
+ char *offs;
+
+ if (field_str[0] == '(')
+ return wprobe_trigger_typecast_parse(field_str, file, wprobe_data, glob);
+
+ offs = strpbrk(field_str, "+-");
+ if (offs) {
+ long val;
+
+ if (kstrtol(offs, 0, &val) < 0) {
+ wprobe_trigger_log_err(file, glob, offs - glob, BAD_DEREF_OFFS);
+ return -EINVAL;
+ }
+ wprobe_data->adjust = val;
+ *offs = '\0';
+ }
+
+ field = trace_find_event_field(file->event_call, field_str);
+ if (!field) {
+ wprobe_trigger_log_err(file, glob, field_str - glob, NO_EVENT_FIELD);
+ return -ENOENT;
+ }
+ if (field->size != sizeof(void *)) {
+ wprobe_trigger_log_err(file, glob, field_str - glob, WPROBE_BAD_FIELD);
+ return -ENOEXEC;
+ }
+ wprobe_data->offset = field->offset;
+ wprobe_data->field = kstrdup(field_str, GFP_KERNEL);
+ if (!wprobe_data->field)
+ return -ENOMEM;
+
+ return 0;
+}
+
+static int wprobe_trigger_cmd_parse(struct event_command *cmd_ops,
+ struct trace_event_file *file,
+ char *glob, char *cmd,
+ char *param_and_filter)
+{
+ /*
+ * set_wprobe:EVENT:FIELD[+OFFS]
+ * clear_wprobe:EVENT[:FIELD[+OFFS]]
+ */
+ struct wprobe_trigger_data *wprobe_data = NULL;
+ struct event_trigger_data *trigger_data = NULL;
+ struct trace_event_file *wprobe_file;
+ char *event_str, *comment;
+ struct trace_array *tr = file->tr;
+ bool remove, clear = false;
+ unsigned long orig_addr = 0;
+ struct trace_wprobe *tw;
+ char *param, *filter;
+ int ret;
+
+ remove = event_trigger_check_remove(glob);
+
+ if (!strcmp(cmd, CLEAR_WPROBE_STR))
+ clear = true;
+
+ if (param_and_filter) {
+ /* Recover original trigger string to show the error log correctly. */
+ if (*(param_and_filter - 1) == '\0')
+ *(param_and_filter - 1) = ':';
+ comment = strchr(param_and_filter, '#');
+ if (comment)
+ *comment = '\0';
+ }
+
+ if (event_trigger_empty_param(param_and_filter)) {
+ wprobe_trigger_log_err(file, glob, strlen(cmd) + 1, WPROBE_NOT_FOUND);
+ return -EINVAL;
+ }
+
+ ret = event_trigger_separate_filter(param_and_filter, ¶m, &filter, true);
+ if (ret)
+ return ret;
+
+ if (file->event_call->flags & TRACE_EVENT_FL_KPROBE) {
+ wprobe_trigger_log_err(file, glob, 0, WPROBE_ON_KPROBE);
+ return -EOPNOTSUPP;
+ }
+
+ event_str = strsep(¶m, ":");
+
+ /* Find target wprobe */
+ tw = find_trace_wprobe(event_str, WPROBE_EVENT_SYSTEM);
+ if (!tw) {
+ wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND);
+ return -ENOENT;
+ }
+ /* The target wprobe must not be used (unless clear) */
+ if (!remove && !clear && trace_probe_is_enabled(&tw->tp)) {
+ wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_BUSY);
+ return -EBUSY;
+ }
+
+ wprobe_file = find_event_file(tr, WPROBE_EVENT_SYSTEM, event_str);
+ if (!wprobe_file) {
+ wprobe_trigger_log_err(file, glob, event_str - glob, WPROBE_NOT_FOUND);
+ return -EINVAL;
+ }
+
+ wprobe_data = wprobe_trigger_alloc(tw, wprobe_file, clear);
+ if (!wprobe_data)
+ return -ENOMEM;
+
+ /* clear_wprobe does not need field, but can have optional field. */
+ if (!clear) {
+ char *field_str = strsep(¶m, ":");
+
+ if (!field_str) {
+ wprobe_trigger_log_err(file, glob, strlen(glob), WPROBE_NEED_FIELD);
+ ret = -EINVAL;
+ goto out_free;
+ }
+ ret = wprobe_trigger_field_parse(field_str, file, wprobe_data, glob);
+ if (ret < 0)
+ goto out_free;
+ } else if (param && (isalpha(param[0]) || param[0] == '_' || param[0] == '(')) {
+ if (strncmp(param, "count=", 6) != 0 &&
+ (strcmp(param, "unlimited") != 0 ||
+ trace_find_event_field(file->event_call, "unlimited"))) {
+ char *field_str = strsep(¶m, ":");
+
+ ret = wprobe_trigger_field_parse(field_str, file, wprobe_data, glob);
+ if (ret < 0)
+ goto out_free;
+ }
+ }
+
+ if (param) {
+ if (!strcmp(param, "unlimited"))
+ param = NULL;
+ else if (!strncmp(param, "count=", 6))
+ param += 6;
+ }
+
+ trigger_data = trigger_data_alloc(cmd_ops, cmd, param, wprobe_data);
+ if (!trigger_data) {
+ ret = -ENOMEM;
+ goto out_free;
+ }
+
+ trigger_data->private_data_free = free_private_wprobe_trigger_data;
+
+ if (remove) {
+ event_trigger_unregister(cmd_ops, file, glob+1, trigger_data);
+ trigger_data_free(trigger_data);
+ return 0;
+ }
+
+ /* Up the trigger_data count to make sure nothing frees it on failure */
+ event_trigger_init(trigger_data);
+
+ ret = event_trigger_parse_num(param, trigger_data);
+ if (ret) {
+ wprobe_trigger_log_err(file, glob, param - glob, BAD_IMM);
+ goto out_free_trigger;
+ }
+
+ ret = event_trigger_set_filter(cmd_ops, file, filter, trigger_data);
+ if (ret < 0)
+ goto out_free_trigger;
+
+ /* Soft-enable (register) wprobe event on WPROBE_DEFAULT_CLEAR_ADDRESS */
+ if (!trace_event_try_get_ref(wprobe_file->event_call)) {
+ ret = -ENODEV;
+ goto out_free_trigger;
+ }
+
+ if (!clear) {
+ orig_addr = tw->addr;
+ WRITE_ONCE(tw->addr, WPROBE_DEFAULT_CLEAR_ADDRESS);
+ }
+
+ ret = trace_event_enable_disable(wprobe_file, 1, 1);
+ if (ret < 0) {
+ if (!clear)
+ WRITE_ONCE(tw->addr, orig_addr);
+ goto out_put;
+ }
+
+ ret = event_trigger_register(cmd_ops, file, glob, trigger_data);
+ if (ret)
+ goto out_disable;
+
+ /* Balance the event_trigger_init() at registration start */
+ event_trigger_free(trigger_data);
+ return 0;
+
+out_disable:
+ if (!clear)
+ WRITE_ONCE(tw->addr, orig_addr);
+ trace_event_enable_disable(wprobe_file, 0, 1);
+out_put:
+ trace_event_put_ref(wprobe_file->event_call);
+out_free_trigger:
+ event_trigger_reset_filter(cmd_ops, trigger_data);
+ trigger_data_free(trigger_data);
+ return ret;
+
+out_free:
+ free_wprobe_trigger_data(wprobe_data);
+ return ret;
+}
+
+/* Return event_trigger_data if there is a trigger which points the same wprobe */
+static struct event_trigger_data *
+wprobe_trigger_find_same(struct event_trigger_data *test,
+ struct trace_event_file *file)
+{
+ struct wprobe_trigger_data *test_wprobe_data = test->private_data;
+ struct wprobe_trigger_data *wprobe_data;
+ struct event_trigger_data *iter;
+
+ list_for_each_entry(iter, &file->triggers, list) {
+ wprobe_data = iter->private_data;
+ if (!wprobe_data ||
+ iter->cmd_ops->trigger_type !=
+ test->cmd_ops->trigger_type)
+ continue;
+ if (wprobe_data->tw == test_wprobe_data->tw &&
+ wprobe_data->clear == test_wprobe_data->clear)
+ return iter;
+ }
+ return NULL;
+}
+
+static int wprobe_register_trigger(char *glob,
+ struct event_trigger_data *data,
+ struct trace_event_file *file)
+{
+ int ret = 0;
+
+ lockdep_assert_held(&event_mutex);
+
+ /* The same wprobe is not accept on the same file (event) */
+ if (wprobe_trigger_find_same(data, file))
+ return -EEXIST;
+
+ if (data->cmd_ops->init) {
+ ret = data->cmd_ops->init(data);
+ if (ret < 0)
+ return ret;
+ }
+
+ list_add_rcu(&data->list, &file->triggers);
+
+ update_cond_flag(file);
+ ret = trace_event_trigger_enable_disable(file, 1);
+ if (ret < 0) {
+ list_del_rcu(&data->list);
+ update_cond_flag(file);
+ tracepoint_synchronize_unregister();
+ if (data->cmd_ops->free)
+ data->cmd_ops->free(data);
+ }
+ return ret;
+}
+
+static void wprobe_unregister_trigger(char *glob,
+ struct event_trigger_data *test,
+ struct trace_event_file *file)
+{
+ struct event_trigger_data *data;
+
+ lockdep_assert_held(&event_mutex);
+
+ data = wprobe_trigger_find_same(test, file);
+ if (!data)
+ return;
+
+ list_del_rcu(&data->list);
+ trace_event_trigger_enable_disable(file, 0);
+ update_cond_flag(file);
+ tracepoint_synchronize_unregister();
+ if (data->cmd_ops->free)
+ data->cmd_ops->free(data);
+}
+
+static struct event_command trigger_wprobe_set_cmd = {
+ .name = SET_WPROBE_STR,
+ .trigger_type = ETT_EVENT_WPROBE,
+ /* This triggers after when the event is recorded. */
+ .flags = EVENT_CMD_FL_NEEDS_REC,
+ .parse = wprobe_trigger_cmd_parse,
+ .reg = wprobe_register_trigger,
+ .unreg = wprobe_unregister_trigger,
+ .set_filter = set_trigger_filter,
+ .trigger = wprobe_trigger,
+ .count_func = wprobe_count_func,
+ .print = wprobe_trigger_print,
+ .init = event_trigger_init,
+ .free = wprobe_trigger_free,
+};
+
+static struct event_command trigger_wprobe_clear_cmd = {
+ .name = CLEAR_WPROBE_STR,
+ .trigger_type = ETT_EVENT_WPROBE,
+ /* This triggers after when the event is recorded. */
+ .flags = EVENT_CMD_FL_NEEDS_REC,
+ .parse = wprobe_trigger_cmd_parse,
+ .reg = wprobe_register_trigger,
+ .unreg = wprobe_unregister_trigger,
+ .set_filter = set_trigger_filter,
+ .trigger = wprobe_trigger,
+ .count_func = wprobe_count_func,
+ .print = wprobe_trigger_print,
+ .init = event_trigger_init,
+ .free = wprobe_trigger_free,
+};
+
+static __init int init_trigger_wprobe_cmds(void)
+{
+ int ret;
+
+ ret = register_event_command(&trigger_wprobe_set_cmd);
+ if (WARN_ON(ret < 0))
+ return ret;
+ ret = register_event_command(&trigger_wprobe_clear_cmd);
+ if (WARN_ON(ret < 0))
+ unregister_event_command(&trigger_wprobe_set_cmd);
+
+ return ret;
+}
+fs_initcall(init_trigger_wprobe_cmds);
+#endif /* CONFIG_WPROBE_TRIGGERS */
diff --git a/tools/testing/selftests/ftrace/config b/tools/testing/selftests/ftrace/config
index 544de0db5f58f..f067874902ed7 100644
--- a/tools/testing/selftests/ftrace/config
+++ b/tools/testing/selftests/ftrace/config
@@ -1,5 +1,6 @@
CONFIG_BPF_SYSCALL=y
CONFIG_DEBUG_INFO_BTF=y
+CONFIG_DEBUG_INFO_BTF_MODULES=y
CONFIG_DEBUG_INFO_DWARF4=y
CONFIG_EPROBE_EVENTS=y
CONFIG_FPROBE=y
@@ -27,3 +28,5 @@ CONFIG_STACK_TRACER=y
CONFIG_TRACER_SNAPSHOT=y
CONFIG_UPROBES=y
CONFIG_UPROBE_EVENTS=y
+CONFIG_WPROBE_EVENTS=y
+CONFIG_WPROBE_TRIGGERS=y
diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
new file mode 100644
index 0000000000000..3c622fb07e7f9
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_wprobe.tc
@@ -0,0 +1,66 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Generic dynamic event - add/remove wprobe events
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README
+
+echo 0 > events/enable
+echo > dynamic_events
+
+# Use jiffies as a variable that is frequently written to.
+TARGET=jiffies
+
+echo "w:my_wprobe w@$TARGET" >> dynamic_events
+
+if ! grep -q my_wprobe dynamic_events; then
+ echo "Failed to create wprobe event"
+ exit_fail
+fi
+
+if [ ! -d events/wprobes/my_wprobe ]; then
+ echo "Failed to create wprobe event directory"
+ exit_fail
+fi
+
+if ! echo 1 > events/wprobes/my_wprobe/enable; then
+ echo "Failed to enable wprobe (hardware watchpoints may be exhausted)"
+ exit_unresolved
+fi
+
+# Check if the event is enabled
+if ! grep -q 1 events/wprobes/my_wprobe/enable; then
+ echo "Failed to enable wprobe event"
+ exit_fail
+fi
+
+# Let some time pass to trigger the breakpoint
+sleep 1
+
+# Check if we got any trace output
+if ! grep -q my_wprobe trace; then
+ echo "wprobe event was not triggered"
+ exit_fail
+fi
+
+echo 0 > events/wprobes/my_wprobe/enable
+
+# Check if the event is disabled
+if ! grep -q 0 events/wprobes/my_wprobe/enable; then
+ echo "Failed to disable wprobe event"
+ exit_fail
+fi
+
+echo "-:my_wprobe" >> dynamic_events
+
+if grep -q my_wprobe dynamic_events; then
+ echo "Failed to remove wprobe event"
+ exit_fail
+fi
+
+if [ -d events/wprobes/my_wprobe ]; then
+ echo "Failed to remove wprobe event directory"
+ exit_fail
+fi
+
+clear_trace
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
new file mode 100644
index 0000000000000..e0df58fe7aa3e
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/dynevent/wprobes_syntax_errors.tc
@@ -0,0 +1,23 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: Watch probe event parser error log check
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README
+
+check_error() { # command-with-error-pos-by-^
+ ftrace_errlog_check 'wprobe' "$1" 'dynamic_events'
+}
+
+check_error '^w' # NO_ARG_BODY
+check_error 'w^10 w@jiffies' # WPROBE_NO_MAXACT
+check_error 'w ^symbol' # BAD_ACCESS_FMT
+check_error 'w ^a@symbol' # BAD_ACCESS_TYPE
+check_error 'w w@^symbol' # BAD_ACCESS_ADDR
+check_error 'w w@^0:4' # BAD_ACCESS_ADDR
+check_error 'w w@jiffies^+offset' # BAD_PROBE_ADDR
+check_error 'w w@jiffies:^100' # BAD_ACCESS_LEN
+check_error 'w w@jiffies ^$arg1' # BAD_VAR
+check_error 'w w@jiffies ^$retval' # BAD_VAR
+check_error 'w w@jiffies ^$stack' # BAD_VAR
+check_error 'w w@jiffies ^%ax' # BAD_VAR
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
new file mode 100644
index 0000000000000..8bc8ca02a712e
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-offset.tc
@@ -0,0 +1,85 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test set_wprobe trigger with BTF struct offset
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event watching 8 bytes" ;:
+echo 'w:watch rw@-1:8 address=$addr value=$value' >> dynamic_events
+
+:;: "Add fprobe event for sample_timer_cb" ;:
+# sample_timer_cb(struct timer_list *t)
+# container_of(t, struct foo_timer_data, timer)
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+
+:;: "Set set_wprobe trigger using BTF struct offset resolution" ;:
+# Syntax: set_wprobe:WPEVENT:(STRUCT,FIELD)EVENT_FIELD->MEMBER
+# (foo_timer_data,timer) is the BTF struct type and field name
+# timer->expires is the struct member whose offset is resolved automatically via BTF
+echo 'set_wprobe:watch:(foo_timer_data,timer)timer->timer.expires' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate
+sleep 3
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+# Since we don't know actual offset of timer->expires in foo_timer_data, we use reset_trigger_file
+reset_trigger_file events/fprobes/testevent/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
new file mode 100644
index 0000000000000..3d4c9fd6b8e23
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-btf-typecast.tc
@@ -0,0 +1,85 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger with BTF typecast fetchargs
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event with BTF typecast fetchargs" ;:
+# (foo_timer_data,timer)$addr->counter reads counter from struct foo_timer_data via BTF typecast
+echo 'w:watch rw@-1:8 address=$addr counter=(foo_timer_data,timer)$addr->counter' >> dynamic_events
+
+:;: "Check the wprobe event is registered with counter field" ;:
+if ! grep -q "counter" dynamic_events; then
+ fail "Failed to register counter field in dynamic_events"
+fi
+
+:;: "Add fprobe event for sample_timer_cb" ;:
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+
+:;: "Set set_wprobe trigger on testevent" ;:
+echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate
+sleep 3
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc
new file mode 100644
index 0000000000000..f82b13d813db1
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-hotplug.tc
@@ -0,0 +1,150 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger across cpu hotplug
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+if ! which nproc >/dev/null 2>&1 ; then
+ nproc() {
+ ls -d /sys/devices/system/cpu/cpu[0-9]* | wc -l
+ }
+fi
+
+NP=`nproc`
+if [ $NP -le 1 ] ; then
+ echo "We cannot test cpu hotplug in UP environment"
+ exit_unresolved
+fi
+
+# Find an online CPU that can be offlined
+for i in /sys/devices/system/cpu/cpu[1-9]*; do
+ if [ -f $i/online ] && [ "$(cat $i/online)" = "1" ]; then
+ cpu=$i
+ break
+ fi
+done
+
+if [ -z "$cpu" ]; then
+ echo "We cannot test cpu hotplug without a hotpluggable online cpu"
+ exit_unresolved
+fi
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ [ -n "$cpu" ] && [ -f "$cpu/online" ] && echo 1 > $cpu/online || true
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then
+ reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event used by trigger" ;:
+echo 'w:watch rw@-1:8 address=$addr value=$value' > dynamic_events
+
+:;: "Add events for triggering wprobe" ;:
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+echo 1 >> events/sample-trace/foo_bar_with_fn/enable
+
+:;: "Set set_wprobe trigger on testevent" ;:
+echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+:;: "Take target CPU offline" ;:
+echo 0 > $cpu/online
+sleep 1
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate while CPU is offline
+sleep 2
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint while CPU is offline"
+fi
+
+:;: "Bring target CPU back online" ;:
+echo 1 > $cpu/online
+sleep 1
+
+# Clear trace and verify that watchpoint triggers after CPU is onlined
+clear_trace
+sleep 2
+
+:;: "Verify watchpoint triggers after CPU came back online" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint after CPU was onlined"
+fi
+
+:;: "Set clear_wprobe trigger on foo_bar_with_fn" ;:
+echo 'clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to set clear_wprobe trigger"
+fi
+
+# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpoint
+echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+
+# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate watchpoint
+sleep 2
+
+# Clear trace and wait to ensure no new watchpoint events are generated
+clear_trace
+sleep 1
+
+:;: "Ensure clear_wprobe trigger deactivated the watchpoint" ;:
+if grep -q watch trace; then
+ fail "Failed to clear watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+echo '!clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to remove clear_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
new file mode 100644
index 0000000000000..dd1ce4baa62be
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe-syntax-errors.tc
@@ -0,0 +1,37 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger syntax errors
+# requires: dynamic_events error_log "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger
+
+check_error() { # command-with-error-pos-by-^
+ ftrace_errlog_check "wprobe_trigger" "$1" "events/fprobes/testevent/trigger"
+}
+
+TARGET_FUNC=$(grep -m 1 -E -w "[tT] (vfs_read|do_sys_openat2|do_sys_open)" /proc/kallsyms | awk '{print $3}')
+if [ -z "$TARGET_FUNC" ]; then
+ echo "UNRESOLVED: target function not found"
+ exit_unresolved
+fi
+
+# Add a dummy fprobe event to attach triggers to
+echo "f:fprobes/testevent $TARGET_FUNC dfd=\$arg1" > dynamic_events
+
+# Add a target wprobe event
+echo 'w:watch rw@-1:8' >> dynamic_events
+
+# Test errors on trigger syntax
+check_error 'set_wprobe:^non_exist_wprobe:dfd' # WPROBE_NOT_FOUND
+check_error 'set_wprobe:^' # WPROBE_NOT_FOUND
+check_error 'set_wprobe:watch^' # WPROBE_NEED_FIELD
+check_error 'set_wprobe:watch:^non_exist_field' # NO_EVENT_FIELD
+
+# Enable target wprobe event and test WPROBE_BUSY error
+echo 1 > events/wprobes/watch/enable
+check_error 'set_wprobe:^watch:dfd' # WPROBE_BUSY
+echo 0 > events/wprobes/watch/enable
+
+# Cleanup
+echo '-:watch' >> dynamic_events
+echo '-:testevent' >> dynamic_events
+
+exit 0
diff --git a/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
new file mode 100644
index 0000000000000..043e1c915ccf1
--- /dev/null
+++ b/tools/testing/selftests/ftrace/test.d/trigger/trigger-wprobe.tc
@@ -0,0 +1,107 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# description: event trigger - test wprobe trigger
+# requires: dynamic_events "w[:[<group>/][<event>]] [r|w|rw]@<addr>[:<len>]":README "f[:[<group>/][<event>]] <func-name>[%return] [<args>]":README events/sched/sched_process_fork/trigger "[(structname[,field])]<argname>[->field[->field|.field...]]":README
+
+fail() { #msg
+ echo "$1"
+ exit_fail
+}
+
+rmmod trace-events-sample ||:
+if ! modprobe trace-events-sample ; then
+ echo "No trace-events sample module - please make CONFIG_SAMPLE_TRACE_EVENTS=m"
+ exit_unresolved
+fi
+
+cleanup_wprobe_triggers() {
+ if [ -f events/fprobes/testevent/trigger ]; then
+ reset_trigger_file events/fprobes/testevent/trigger || true
+ fi
+ if [ -f events/sample-trace/foo_bar_with_fn/trigger ]; then
+ reset_trigger_file events/sample-trace/foo_bar_with_fn/trigger || true
+ fi
+ echo 0 > events/enable 2>/dev/null || true
+ echo > dynamic_events 2>/dev/null || true
+ sleep 1
+ rmmod trace-events-sample 2>/dev/null || true
+ return 0
+}
+
+trap cleanup_wprobe_triggers EXIT
+
+echo 0 > tracing_on
+
+# we will skip this test if fprobe is not supported.
+if ! grep -Fq "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README; then
+ echo "UNRESOLVED: fprobe is not supported"
+ exit_unresolved
+fi
+
+# we will skip this test if the target function does not exist.
+if ! grep -wq "sample_timer_cb" /proc/kallsyms; then
+ echo "UNRESOLVED: sample_timer_cb not found"
+ exit_unresolved
+fi
+
+:;: "Add a wprobe event used by trigger" ;:
+echo 'w:watch rw@-1:8 address=$addr value=$value' > dynamic_events
+
+:;: "Add events for triggering wprobe" ;:
+echo 'f:fprobes/testevent sample_timer_cb timer=t' >> dynamic_events
+
+:;: "Enable all events before setting triggers" ;:
+echo 1 > tracing_on
+echo 1 >> events/fprobes/testevent/enable
+echo 1 >> events/sample-trace/foo_bar_with_fn/enable
+
+:;: "Set set_wprobe trigger on testevent" ;:
+echo 'set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+if ! grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to set set_wprobe trigger"
+fi
+
+# Wait for sample_timer_cb to fire and set_wprobe trigger to activate
+sleep 2
+
+:;: "Check set_wprobe trigger activated the watchpoint" ;:
+if ! grep -q watch trace; then
+ fail "Failed to trigger watchpoint"
+fi
+
+:;: "Set clear_wprobe trigger on foo_bar_with_fn" ;:
+echo 'clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if ! grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to set clear_wprobe trigger"
+fi
+
+# Disable set_wprobe to prevent sample_timer_cb from re-arming the watchpoint
+echo '!set_wprobe:watch:timer' >> events/fprobes/testevent/trigger
+
+# Wait for foo_bar_with_fn to fire and clear_wprobe trigger to deactivate watchpoint
+sleep 2
+
+# Clear trace and wait to ensure no new watchpoint events are generated
+clear_trace
+sleep 1
+
+:;: "Ensure clear_wprobe trigger deactivated the watchpoint" ;:
+if grep -q watch trace; then
+ fail "Failed to clear watchpoint"
+fi
+
+:;: "Remove wprobe triggers" ;:
+echo '!clear_wprobe:watch' >> events/sample-trace/foo_bar_with_fn/trigger
+if grep -q ^set_wprobe events/fprobes/testevent/trigger; then
+ fail "Failed to remove set_wprobe trigger"
+fi
+if grep -q ^clear_wprobe events/sample-trace/foo_bar_with_fn/trigger; then
+ fail "Failed to remove clear_wprobe trigger"
+fi
+
+:;: "Disable events and remove dynamic events" ;:
+echo 0 > events/enable
+echo > dynamic_events
+clear_trace
+
+exit 0
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 65eb26a0520d8..7779d500fe774 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -6476,11 +6476,23 @@ static unsigned long kvm_guest_get_ip(void)
return kvm_arch_vcpu_get_ip(vcpu);
}
+static bool kvm_guest_in_guest(void)
+{
+ struct kvm_vcpu *vcpu = kvm_get_running_vcpu();
+
+ if (!vcpu)
+ return false;
+
+ return READ_ONCE(vcpu->mode) == IN_GUEST_MODE ||
+ READ_ONCE(vcpu->mode) == EXITING_GUEST_MODE;
+}
+
static struct perf_guest_info_callbacks kvm_guest_cbs = {
.state = kvm_guest_state,
.get_ip = kvm_guest_get_ip,
.handle_intel_pt_intr = NULL,
.handle_mediated_pmi = NULL,
+ .in_guest = kvm_guest_in_guest,
};
void __kvm_register_perf_callbacks(unsigned int (*pt_intr_handler)(void),
|
| 13/3 | 2026/09/24 05:45 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 45979 output: 30 thoughts: 1169 Thoughts: **KMSAN Fuzzing Session Assessment** Okay, let's break this patch series down and figure out if it warrants a KMSAN fuzzing session. The primary goal here is to determine if this code introduces any new risks related to uninitialized memory that *only* KMSAN would catch. Other bug detectors should be able to catch bugs earlier. First, I see the patch adds documentation, Kconfig options, and significant changes to x86-specific hardware breakpoint handling, along with introducing watchpoint probe tracing. Specifically, it seems the patch implements a way to trace execution that is triggered by hardware breakpoints. This sounds interesting and complex, so I need to go over it with a fine-toothed comb. Now I need to zero in on areas where uninitialized memory could become a problem. Based on the rules, my main concerns are: 1. Kernel structures passed to userspace (copy\_to\_user, etc.). 2. Conditional logic based on potentially uninitialized variables. 3. Complex data structure allocation and initialization. 4. Bounds checks and potential access to uninitialized buffer bytes. Let's start with `kernel/trace/trace_wprobe.c` and other related files. It seems the code defines a `wprobe_trace_entry_head` struct, which contains a `trace_entry` and an `ip`. The critical path to investigate involves how this `struct` is written to a ring buffer and eventually read by userspace. The handler `wprobe_trace_handler` is where the action happens. I see `trace_event_buffer_reserve` is being called to allocate memory for the trace entry. Then the function populates `entry->ip`, and the function `store_trace_args` is called. The question is, does `trace_event_buffer_reserve` actually initialize `entry->ent`? This could be a problem if it doesn't, since `trace_entry` is part of the struct. This seems like it should be the key to triggering KMSAN. Let's use `grepper` to look into how `trace_event_buffer_reserve` actually works and whether it zero initializes allocated memory, or just allocates memory and leaves it as is. |
| 14/3 | 2026/09/24 05:45 | tool | grepper |
0mArgs:{
"Expression": "trace_event_buffer_reserve",
"PathPrefix": "kernel/trace"
}Results: {
"Output": "kernel/trace/trace_eprobe.c=428=__eprobe_trace_func(struct eprobe_data *edata, void *rec)\n--\nkernel/trace/trace_eprobe.c-442-\nkernel/trace/trace_eprobe.c:443:\tentry = trace_event_buffer_reserve(\u0026fbuffer, edata-\u003efile,\nkernel/trace/trace_eprobe.c-444-\t\t\t\t\t sizeof(*entry) + edata-\u003eep-\u003etp.size + dsize);\n--\nkernel/trace/trace_events.c=677=EXPORT_SYMBOL_GPL(trace_event_ignore_this_pid);\n--\nkernel/trace/trace_events.c-679-/**\nkernel/trace/trace_events.c:680: * trace_event_buffer_reserve - reserve space on the ring buffer for an event\nkernel/trace/trace_events.c-681- * @fbuffer: information about how to save the event\n--\nkernel/trace/trace_events.c-694- */\nkernel/trace/trace_events.c:695:void *trace_event_buffer_reserve(struct trace_event_buffer *fbuffer,\nkernel/trace/trace_events.c-696-\t\t\t\t struct trace_event_file *trace_file,\n--\nkernel/trace/trace_events.c-724-}\nkernel/trace/trace_events.c:725:EXPORT_SYMBOL_GPL(trace_event_buffer_reserve);\nkernel/trace/trace_events.c-726-\n--\nkernel/trace/trace_events_inject.c=17=trace_inject_entry(struct trace_event_file *file, void *rec, int len)\n--\nkernel/trace/trace_events_inject.c-23-\trcu_read_lock_sched();\nkernel/trace/trace_events_inject.c:24:\tentry = trace_event_buffer_reserve(\u0026fbuffer, file, len);\nkernel/trace/trace_events_inject.c-25-\tif (entry) {\n--\nkernel/trace/trace_events_synth.c=583=static void trace_event_raw_event_synth(void *__data,\n--\nkernel/trace/trace_events_synth.c-607-\nkernel/trace/trace_events_synth.c:608:\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file,\nkernel/trace/trace_events_synth.c-609-\t\t\t\t\t sizeof(*entry) + fields_size);\n--\nkernel/trace/trace_events_synth.c=1755=__synth_event_trace_start(struct trace_event_file *file,\n--\nkernel/trace/trace_events_synth.c-1772-\tentry_size = sizeof(*trace_state-\u003eentry) + fields_size;\nkernel/trace/trace_events_synth.c:1773:\ttrace_state-\u003eentry = trace_event_buffer_reserve(\u0026trace_state-\u003efbuffer,\nkernel/trace/trace_events_synth.c-1774-\t\t\t\t\t\t\tfile,\n--\nkernel/trace/trace_events_user.c=1608=static void user_event_ftrace(struct user_event *user, struct iov_iter *i,\n--\nkernel/trace/trace_events_user.c-1623-\t/* Allocates and fills trace_entry, + 1 of this is data payload */\nkernel/trace/trace_events_user.c:1624:\tentry = trace_event_buffer_reserve(\u0026event_buffer, file, size);\nkernel/trace/trace_events_user.c-1625-\n--\nkernel/trace/trace_fprobe.c=318=__fentry_trace_func(struct trace_fprobe *tf, unsigned long entry_ip,\n--\nkernel/trace/trace_fprobe.c-334-\nkernel/trace/trace_fprobe.c:335:\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file,\nkernel/trace/trace_fprobe.c-336-\t\t\t\t\t sizeof(*entry) + tf-\u003etp.size + dsize);\n--\nkernel/trace/trace_fprobe.c=404=__fexit_trace_func(struct trace_fprobe *tf, unsigned long entry_ip,\n--\nkernel/trace/trace_fprobe.c-420-\nkernel/trace/trace_fprobe.c:421:\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file,\nkernel/trace/trace_fprobe.c-422-\t\t\t\t\t sizeof(*entry) + tf-\u003etp.size + dsize);\n--\nkernel/trace/trace_kprobe.c=1486=__kprobe_trace_func(struct trace_kprobe *tk, struct pt_regs *regs,\n--\nkernel/trace/trace_kprobe.c-1500-\nkernel/trace/trace_kprobe.c:1501:\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file,\nkernel/trace/trace_kprobe.c-1502-\t\t\t\t\t sizeof(*entry) + tk-\u003etp.size + dsize);\n--\nkernel/trace/trace_kprobe.c=1550=__kretprobe_trace_func(struct trace_kprobe *tk, struct kretprobe_instance *ri,\n--\nkernel/trace/trace_kprobe.c-1565-\nkernel/trace/trace_kprobe.c:1566:\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file,\nkernel/trace/trace_kprobe.c-1567-\t\t\t\t\t sizeof(*entry) + tk-\u003etp.size + dsize);\n--\nkernel/trace/trace_syscalls.c=861=static void ftrace_syscall_enter(void *data, struct pt_regs *regs, long id)\n--\nkernel/trace/trace_syscalls.c-911-\nkernel/trace/trace_syscalls.c:912:\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file, size);\nkernel/trace/trace_syscalls.c-913-\tif (!entry)\n--\nkernel/trace/trace_syscalls.c=927=static void ftrace_syscall_exit(void *data, struct pt_regs *regs, long ret)\n--\nkernel/trace/trace_syscalls.c-957-\nkernel/trace/trace_syscalls.c:958:\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file, sizeof(*entry));\nkernel/trace/trace_syscalls.c-959-\tif (!entry)\n--\nkernel/trace/trace_uprobe.c=997=static void __uprobe_trace_func(struct trace_uprobe *tu,\n--\nkernel/trace/trace_uprobe.c-1014-\tsize = esize + ucb-\u003edsize;\nkernel/trace/trace_uprobe.c:1015:\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file, size);\nkernel/trace/trace_uprobe.c-1016-\tif (!entry)\n--\nkernel/trace/trace_wprobe.c=149=static void wprobe_trace_handler(struct trace_wprobe *tw,\n--\nkernel/trace/trace_wprobe.c-169-\nkernel/trace/trace_wprobe.c:170:\tentry = trace_event_buffer_reserve(\u0026fbuffer, trace_file,\nkernel/trace/trace_wprobe.c-171-\t\t\t\t\t sizeof(*entry) + tw-\u003etp.size + dsize);\n"
} |
| 15/3 | 2026/09/24 05:45 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 49554 output: 37 thoughts: 26 Thoughts: Okay, let's dive in. **Inspecting `trace_event_buffer_reserve` Implementation** Alright, I need to get a handle on how this `trace_event_buffer_reserve` function actually works. Time to crack open the source code and take a look at its implementation, specifically in `kernel/trace/trace_events.c`. I need to understand its behavior, how it interacts with the buffer, and any potential areas of concern for performance or synchronization. This is where the rubber meets the road, so let's dig in and see what's what. |
| 16/3 | 2026/09/24 05:45 | tool | read-file |
0mArgs:{
"File": "kernel/trace/trace_events.c",
"FirstLine": 695,
"LineCount": 35
}Results: {
"Contents": " 695:\tvoid *trace_event_buffer_reserve(struct trace_event_buffer *fbuffer,\n 696:\t\t\t\t\t struct trace_event_file *trace_file,\n 697:\t\t\t\t\t unsigned long len)\n 698:\t{\n 699:\t\tstruct trace_event_call *event_call = trace_file-\u003eevent_call;\n 700:\t\n 701:\t\tif ((trace_file-\u003eflags \u0026 EVENT_FILE_FL_PID_FILTER) \u0026\u0026\n 702:\t\t trace_event_ignore_this_pid(trace_file))\n 703:\t\t\treturn NULL;\n 704:\t\n 705:\t\t/*\n 706:\t\t * If CONFIG_PREEMPTION is enabled, then the tracepoint itself disables\n 707:\t\t * preemption (adding one to the preempt_count). Since we are\n 708:\t\t * interested in the preempt_count at the time the tracepoint was\n 709:\t\t * hit, we need to subtract one to offset the increment.\n 710:\t\t */\n 711:\t\tfbuffer-\u003etrace_ctx = tracing_gen_ctx_dec();\n 712:\t\tfbuffer-\u003etrace_file = trace_file;\n 713:\t\n 714:\t\tfbuffer-\u003eevent =\n 715:\t\t\ttrace_event_buffer_lock_reserve(\u0026fbuffer-\u003ebuffer, trace_file,\n 716:\t\t\t\t\t\t\tevent_call-\u003eevent.type, len,\n 717:\t\t\t\t\t\t\tfbuffer-\u003etrace_ctx);\n 718:\t\tif (!fbuffer-\u003eevent)\n 719:\t\t\treturn NULL;\n 720:\t\n 721:\t\tfbuffer-\u003eregs = NULL;\n 722:\t\tfbuffer-\u003eentry = ring_buffer_event_data(fbuffer-\u003eevent);\n 723:\t\treturn fbuffer-\u003eentry;\n 724:\t}\n 725:\tEXPORT_SYMBOL_GPL(trace_event_buffer_reserve);\n 726:\t\n 727:\tint trace_event_reg(struct trace_event_call *call,\n 728:\t\t\t enum trace_reg type, void *data)\n 729:\t{\n"
} |
| 17/3 | 2026/09/24 05:45 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 50303 output: 32 thoughts: 905 Thoughts: **Analyzing the Trace Probe Pattern and Debug Register Handling**
Okay, so I've been looking at how this `wprobe_trace_handler` is set up. First, I noticed that `trace_event_buffer_lock_reserve` is initializing `trace_entry`. I'm going to quickly check how the other probes handle this, particularly in `ring_buffer_event_data`. Right, looking at `trace_kprobe.c`, `trace_fprobe.c`, and `trace_eprobe.c`, they're all following a standard pattern: `trace_event_buffer_reserve`, setting `entry->ip`, then `store_trace_args`, and finally `trace_event_buffer_commit`. So, `wprobe_trace_handler` is indeed following the expected trace probe structure.
Now, let's dive into these changes more deeply. I'm focusing on the debug register handling. I'm starting with `arch/x86/include/asm/debugreg.h` and the `local_db_save` function. I see this:
```c
static __always_inline void local_db_save(unsigned long *dr7,
unsigned int *dr7_seq)
{
bool retrying = false;
do {
/* Inner loop: get DR7 with checking cpu_dr7_seq. */
*dr7_seq = this_cpu_read(cpu_dr7_seq);
if (unlikely(retrying)) {
*dr7 = this_cpu_read(cpu_dr7);
} else {
get_debugreg(*dr7, 7);
/* Architecturally set bit */
*dr7 &= ~DR7_FIXED_1;
}
barrier();
if (unlikely(*dr7_seq != this_cpu_read(cpu_dr7_seq))) {
retrying = true;
continue;
}
/* Outer loop: clear DR7 and retry if sequence number is updated. */
if (*dr7)
set_debugreg(DR7_FIXED_1, 7);
/*
* Ensure the compiler doesn't lower the above statements into
* the critical section; disabling breakpoints late would not
* be good.
*/
barrier();
retrying = true;
} while (unlikely(*dr7_seq != this_cpu_read(cpu_dr7_seq)));
}
```
Interesting... I'm quickly checking where `local_db_save(&dr7, &dr7_seq)` is being called. It's in `exc_machine_check_kernel`, `exc_machine_check_user`, `exc_nmi`, and `exc_debug_kernel`. In each case, it looks like `dr7` and `dr7_seq` are passed in, and within the `local_db_save` function, those pointers are immediately written to. And then they're passed to `local_db_restore`. Hmm, the intent there is pretty clear.
Alright, moving on, let's look at `arch_modify_local_hw_breakpoint_addr` in `arch/x86/kernel/hw_breakpoint.c`. There's this:
```c
int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,
unsigned long addr)
{
struct arch_hw_breakpoint *info = counter_arch_bp(bp);
struct arch_hw_breakpoint hw;
struct perf_event_attr attr = bp->attr;
unsigned int seq;
int i, ret;
lockdep_assert_irqs_disabled();
if (perf_guest_in_guest())
return -EBUSY;
attr.bp_addr = addr;
ret = hw_breakpoint_arch_parse(bp, &attr, &hw);
if (ret)
return ret;
...
```
Now, I'm curious about this `hw` variable. How is it being *used* after being populated by `hw_breakpoint_arch_parse`? Is it read? Let me check `hw_breakpoint_arch_parse` itself. I need to see what's happening to that `hw` struct... and if there are other calls to that function. I'll just grep for `hw_breakpoint_arch_parse`.
|
| 18/3 | 2026/09/24 05:45 | tool | grepper |
0mArgs:{
"Expression": "hw_breakpoint_arch_parse",
"PathPrefix": "arch/x86"
}Results: {
"Output": "arch/x86/include/asm/hw_breakpoint.h=53=extern int arch_check_bp_in_kernelspace(struct arch_hw_breakpoint *hw);\narch/x86/include/asm/hw_breakpoint.h:54:extern int hw_breakpoint_arch_parse(struct perf_event *bp,\narch/x86/include/asm/hw_breakpoint.h-55-\t\t\t\t const struct perf_event_attr *attr,\n--\narch/x86/kernel/hw_breakpoint.c=144=int arch_modify_local_hw_breakpoint_addr(struct perf_event *bp,\n--\narch/x86/kernel/hw_breakpoint.c-158-\tattr.bp_addr = addr;\narch/x86/kernel/hw_breakpoint.c:159:\tret = hw_breakpoint_arch_parse(bp, \u0026attr, \u0026hw);\narch/x86/kernel/hw_breakpoint.c-160-\tif (ret)\n--\narch/x86/kernel/hw_breakpoint.c=382=static int arch_build_bp_info(struct perf_event *bp,\n--\narch/x86/kernel/hw_breakpoint.c-480- */\narch/x86/kernel/hw_breakpoint.c:481:int hw_breakpoint_arch_parse(struct perf_event *bp,\narch/x86/kernel/hw_breakpoint.c-482-\t\t\t const struct perf_event_attr *attr,\n"
} |
| 19/3 | 2026/09/24 05:45 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 52613 output: 41 thoughts: 29 Thoughts: Okay, let's dive into this. Here's my take on those lines of code: **Analyzing Hardware Breakpoint Handling in the x86 Kernel** Alright, I need to get a handle on how hardware breakpoints are being managed at the kernel level on x86, so I'm honing in on lines 480-500 in `arch/x86/kernel/hw_breakpoint.c`. I'm expecting this section deals with some critical aspect of breakpoint configuration or handling, perhaps the actual mechanism for setting up the debug registers or managing the breakpoint state. Given my experience, I'm likely to focus on the interplay between the debug registers (DR0-DR7), the task state segment (TSS), and any potential locking that might be involved to prevent race conditions when multiple processes are vying for breakpoints. I'll be looking for how the code handles different breakpoint types (instruction execution, data access) and their associated conditions (read, write, execute, address range matching). I'll also be watching for any checks for privileged access, and any potential issues regarding virtualization and nested paging in the hypervisor, especially in regard to the possibility of security vulnerabilities that may result in kernel control. I'm expecting to see calls to functions that likely write to the debug registers, modify the task state segment, or perhaps manipulate some internal data structures related to breakpoint tracking. The devil is always in the details, so I'll be paying close attention to any error handling or edge case management within this crucial section of the code. |
| 20/3 | 2026/09/24 05:45 | tool | read-file |
0mArgs:{
"File": "arch/x86/kernel/hw_breakpoint.c",
"FirstLine": 480,
"LineCount": 20
}Results: {
"Contents": " 480:\t */\n 481:\tint hw_breakpoint_arch_parse(struct perf_event *bp,\n 482:\t\t\t\t const struct perf_event_attr *attr,\n 483:\t\t\t\t struct arch_hw_breakpoint *hw)\n 484:\t{\n 485:\t\tunsigned int align;\n 486:\t\tint ret;\n 487:\t\n 488:\t\n 489:\t\tret = arch_build_bp_info(bp, attr, hw);\n 490:\t\tif (ret)\n 491:\t\t\treturn ret;\n 492:\t\n 493:\t\tswitch (hw-\u003elen) {\n 494:\t\tcase X86_BREAKPOINT_LEN_1:\n 495:\t\t\talign = 0;\n 496:\t\t\tif (hw-\u003emask)\n 497:\t\t\t\talign = hw-\u003emask;\n 498:\t\t\tbreak;\n 499:\t\tcase X86_BREAKPOINT_LEN_2:\n"
} |
| 21/3 | 2026/09/24 05:45 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 53277 output: 26 thoughts: 631 Thoughts: **Analyzing Potential Vulnerabilities in Kernel Tracepoint Implementation** Okay, so I'm digging into this kernel tracepoint code, trying to see if there are any obvious memory safety issues. Let's see... the `hw_breakpoint_arch_parse` function seems fine; it validates and populates the `hw` struct, but that struct isn't even used afterward in `arch_modify_local_hw_breakpoint_addr`, it's just a validation tool. Seems reasonable. Now, let's move onto `alloc_trace_wprobe`. Ah, good, it uses `kzalloc_flex`! That means the `tw` struct is fully zeroed out, which is a good starting point for safety. And `wprobe_trigger_alloc` uses `kzalloc_obj`, also zero-initialized. I like that. Even `__trace_wprobe_create` also initializes the `ctx` struct this way. It allocates with `kmalloc`, but that seems ok because it's only for the event names and such. It does appear that a lot of string manipulation is going on with the event names, using `gbuf` and `ebuf`. This is worth a second look but at first glance, I'm not seeing any red flags from that. Now, hold on. Let's check `wprobe_event_define_fields`... What's this `DEFINE_FIELD` macro doing? That's interesting. I need to understand how the other probe definitions handle this macro. It defines a field and then calls `traceprobe_define_arg_fields`. I need to figure out how these fields get populated and what safeguards are in place. This warrants a more detailed investigation. |
| 22/3 | 2026/09/24 05:45 | tool | grepper |
0mArgs:{
"Expression": "DEFINE_FIELD",
"PathPrefix": "kernel/trace"
}Results: {
"Output": "kernel/trace/trace_fprobe.c=681=static int fentry_event_define_fields(struct trace_event_call *event_call)\n--\nkernel/trace/trace_fprobe.c-690-\nkernel/trace/trace_fprobe.c:691:\tDEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);\nkernel/trace/trace_fprobe.c-692-\n--\nkernel/trace/trace_fprobe.c=696=static int fexit_event_define_fields(struct trace_event_call *event_call)\n--\nkernel/trace/trace_fprobe.c-705-\nkernel/trace/trace_fprobe.c:706:\tDEFINE_FIELD(unsigned long, func, FIELD_STRING_FUNC, 0);\nkernel/trace/trace_fprobe.c:707:\tDEFINE_FIELD(unsigned long, ret_ip, FIELD_STRING_RETIP, 0);\nkernel/trace/trace_fprobe.c-708-\n--\nkernel/trace/trace_kprobe.c=1658=static int kprobe_event_define_fields(struct trace_event_call *event_call)\n--\nkernel/trace/trace_kprobe.c-1667-\nkernel/trace/trace_kprobe.c:1668:\tDEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);\nkernel/trace/trace_kprobe.c-1669-\n--\nkernel/trace/trace_kprobe.c=1673=static int kretprobe_event_define_fields(struct trace_event_call *event_call)\n--\nkernel/trace/trace_kprobe.c-1682-\nkernel/trace/trace_kprobe.c:1683:\tDEFINE_FIELD(unsigned long, func, FIELD_STRING_FUNC, 0);\nkernel/trace/trace_kprobe.c:1684:\tDEFINE_FIELD(unsigned long, ret_ip, FIELD_STRING_RETIP, 0);\nkernel/trace/trace_kprobe.c-1685-\n--\nkernel/trace/trace_probe.h-47-\nkernel/trace/trace_probe.h:48:#undef DEFINE_FIELD\nkernel/trace/trace_probe.h:49:#define DEFINE_FIELD(type, item, name, is_signed)\t\t\t\\\nkernel/trace/trace_probe.h-50-\tdo {\t\t\t\t\t\t\t\t\\\n--\nkernel/trace/trace_uprobe.c=1225=static int uprobe_event_define_fields(struct trace_event_call *event_call)\n--\nkernel/trace/trace_uprobe.c-1235-\tif (is_ret_probe(tu)) {\nkernel/trace/trace_uprobe.c:1236:\t\tDEFINE_FIELD(unsigned long, vaddr[0], FIELD_STRING_FUNC, 0);\nkernel/trace/trace_uprobe.c:1237:\t\tDEFINE_FIELD(unsigned long, vaddr[1], FIELD_STRING_RETIP, 0);\nkernel/trace/trace_uprobe.c-1238-\t\tsize = SIZEOF_TRACE_ENTRY(true);\nkernel/trace/trace_uprobe.c-1239-\t} else {\nkernel/trace/trace_uprobe.c:1240:\t\tDEFINE_FIELD(unsigned long, vaddr[0], FIELD_STRING_IP, 0);\nkernel/trace/trace_uprobe.c-1241-\t\tsize = SIZEOF_TRACE_ENTRY(false);\n--\nkernel/trace/trace_wprobe.c=437=static int wprobe_event_define_fields(struct trace_event_call *event_call)\n--\nkernel/trace/trace_wprobe.c-446-\nkernel/trace/trace_wprobe.c:447:\tDEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);\nkernel/trace/trace_wprobe.c-448-\n"
} |
| 23/3 | 2026/09/24 05:45 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 55295 output: 36 thoughts: 59 Thoughts: Here's my summary, as if I were thinking it: **Verifying `DEFINE_FIELD` Macro Usage** Okay, so I'm looking at this `DEFINE_FIELD` macro, and my initial thought is that it's probably using `offsetof` and `sizeof` under the hood. Specifically, it's taking the `offsetof` of `field.item` (where `field` is likely a struct or class) and the `sizeof` of that same `field.item`. The interesting thing is, it doesn't seem to care *what* the contents of `field` actually *are*; it's just working on the structure's layout. I want to be certain about this, so I need to verify how it's actually implemented. Time to dive into those lines 48-55 of `trace_probe.h`. Let's see if that's what's happening. |
| 24/3 | 2026/09/24 05:45 | tool | read-file |
0mArgs:{
"File": "kernel/trace/trace_probe.h",
"FirstLine": 48,
"LineCount": 15
}Results: {
"Contents": " 48:\t#undef DEFINE_FIELD\n 49:\t#define DEFINE_FIELD(type, item, name, is_signed)\t\t\t\\\n 50:\t\tdo {\t\t\t\t\t\t\t\t\\\n 51:\t\t\tret = trace_define_field(event_call, #type, name,\t\\\n 52:\t\t\t\t\t\t offsetof(typeof(field), item),\t\\\n 53:\t\t\t\t\t\t sizeof(field.item), is_signed, \\\n 54:\t\t\t\t\t\t FILTER_OTHER);\t\t\t\\\n 55:\t\t\tif (ret)\t\t\t\t\t\t\\\n 56:\t\t\t\treturn ret;\t\t\t\t\t\\\n 57:\t\t} while (0)\n 58:\t\n 59:\t\n 60:\t/* Flags for trace_probe */\n 61:\t#define TP_FLAG_TRACE\t\t1\n 62:\t#define TP_FLAG_PROFILE\t\t2\n"
} |
| 25/3 | 2026/09/24 05:45 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 55806 output: 28 thoughts: 1456 Thoughts: **KMSAN Consideration for Wprobes** Okay, let's break this down. I'm looking at how `wprobe` handles `$value`, specifically how it might interact with uninitialized memory and KMSAN. First, the `process_fetch_insn` function – it's key. It appears to retrieve a value, `val`. The logic dictates that when `$value` is used, the code sequence sets `FETCH_OP_BADDR`, which assigns `val` the value of `baddr` (which comes from `rec`), and then `FETCH_OP_DEREF` dereferences that address, effectively reading memory. Crucially, the code reads memory at an address provided by the user. Now, if `addr` contains uninitialized memory, that's a problem. KMSAN is designed to catch uninitialized reads. So, I need to check the KMSAN rules. Am I introducing a scenario where uninitialized memory is read and potentially sent to user space? I remember the KMSAN rules by heart: Rule 1 deals with data structures copied to user space, Rule 2 covers conditional logic on uninitialized data, Rule 3 is around complex data structure initialization, and Rule 4 talks about bounds checks. In the context of the ftrace trace ring buffer, it reads kernel memory and writes it to the trace output file. Does this read of memory, formatted into the trace output file (`trace` or `trace_pipe`), constitute sending a kernel structure to user space? While formatted as text, the *content* is a representation of kernel memory. Let's make sure, however, that even if the output is not sent to user space, the trace data itself is not a kernel structure sent to user space. Checking the `wprobe_trace_handler` and `parse_probe_vars`, I'm looking for where the data is written. It uses `trace_event_buffer_reserve` which creates an `entry` containing trace information. `store_trace_args` is where the data gets marshalled. Are there *any* kernel structures from `wprobe` that are copied to user space? There's no `copy_to_user`, no netlink, no IOCTLs, none of those things. The data ends up in the trace buffer, which is read by userspace via `trace` or `trace_pipe`. The `wprobe_trace_entry_head` struct looks interesting. It has a `trace_entry` and an `ip`. The question is: is there any padding within `struct wprobe_trace_entry_head` that *could* be uninitialized. The important point is the data that `$value` retrieves from kernel memory. Is that *formatted* in any way that makes it a structure sent to user space? It is not a structured message as `netlink` or `ioctl`, it is a memory read at `addr`. To recap: `wprobe` reads kernel memory, that is written into the trace ring buffer, and finally sent in raw memory in a structured data. No explicit copy to userspace. |
| 26/3 | 2026/09/24 05:46 | tool | grepper |
0mArgs:{
"Expression": "struct trace_entry \\{",
"PathPrefix": "include/linux"
}Results: {
"Output": "include/linux/trace_events.h=69=struct trace_dynamic_info {\n--\ninclude/linux/trace_events.h-84- */\ninclude/linux/trace_events.h:85:struct trace_entry {\ninclude/linux/trace_events.h-86-\tunsigned short\t\ttype;\n"
} |
| 27/3 | 2026/09/24 05:46 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 58039 output: 36 thoughts: 22 Thoughts: Okay, let's dive into this. Here's my take on understanding this code: **Analyzing the `struct trace_entry` Definition** Alright, time to get into the heart of the matter. I need to understand how the tracing infrastructure is structured, so I'm heading directly to the source. Let's start with the base structure, `struct trace_entry`, which is defined in `include/linux/trace_events.h`. This is likely the fundamental building block for all trace records, the common denominator. I'll need to see its fields, their data types, and any comments explaining their purpose. I'm expecting this structure to hold crucial information like the timestamp, the CPU ID, the event ID, and possibly a pointer to where the event data resides. Understanding this structure is paramount, because it essentially forms the base unit for all the collected tracing data. This is where I can start building up an understanding of how data is stored, read, and most likely, analyzed. Let's see what the definition actually holds. |
| 28/3 | 2026/09/24 05:46 | tool | read-file |
0mArgs:{
"File": "include/linux/trace_events.h",
"FirstLine": 85,
"LineCount": 15
}Results: {
"Contents": " 85:\tstruct trace_entry {\n 86:\t\tunsigned short\t\ttype;\n 87:\t\tunsigned char\t\tflags;\n 88:\t\tunsigned char\t\tpreempt_count;\n 89:\t\tint\t\t\tpid;\n 90:\t};\n 91:\t\n 92:\t#define TRACE_EVENT_TYPE_MAX\t\t\t\t\t\t\\\n 93:\t\t((1 \u003c\u003c (sizeof(((struct trace_entry *)0)-\u003etype) * 8)) - 1)\n 94:\t\n 95:\t/*\n 96:\t * Trace iterator - used by printout routines who present trace\n 97:\t * results to users and which routines might sleep, etc:\n 98:\t */\n 99:\tstruct trace_iterator {\n"
} |
| 29/3 | 2026/09/24 05:46 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 58513 output: 32 thoughts: 150 Thoughts: Okay, here's my understanding, formatted as requested: **Deconstructing Trace Entry Structures: A Memory Mapping** Alright, let's break down these trace entry structures. First, we have the base `struct trace_entry`. It's pretty straightforward: `type` (2 bytes), `flags` (1 byte), `preempt_count` (1 byte), and `pid` (4 bytes). That's a total of 8 bytes. Excellent, no padding there; we're keeping it tight. This is crucial for performance. Next, we encounter `struct wprobe_trace_entry_head`. This one includes the `struct trace_entry ent;` which we know occupies 8 bytes (offset 0), followed by an `unsigned long ip;` which, on a 64-bit architecture, is also 8 bytes (offset 8). So, the total for `wprobe_trace_entry_head` is 16 bytes. Again, no padding. Good! We need to be efficient with our memory. Finally, we're looking into `struct kentry_trace_entry_head`. I need to go and look it up in `kernel/trace/trace_kprobe.c`, let's see what that includes. I'll need to go and refresh my memory on that, but it should be a variation on the same theme as the others. I'm assuming that it is efficient, just like the others. I assume that it's going to be packed in the same way. |
| 30/3 | 2026/09/24 05:46 | tool | grepper |
0mArgs:{
"Expression": "struct kprobe_trace_entry_head",
"PathPrefix": "kernel/trace"
}Results: {
"Output": "kernel/trace/trace.h=150=struct syscall_trace_exit {\n--\nkernel/trace/trace.h-155-\nkernel/trace/trace.h:156:struct kprobe_trace_entry_head {\nkernel/trace/trace.h-157-\tstruct trace_entry\tent;\n--\nkernel/trace/trace_kprobe.c=1486=__kprobe_trace_func(struct trace_kprobe *tk, struct pt_regs *regs,\n--\nkernel/trace/trace_kprobe.c-1488-{\nkernel/trace/trace_kprobe.c:1489:\tstruct kprobe_trace_entry_head *entry;\nkernel/trace/trace_kprobe.c-1490-\tstruct trace_event_call *call = trace_probe_event_call(\u0026tk-\u003etp);\n--\nkernel/trace/trace_kprobe.c=1592=print_kprobe_event(struct trace_iterator *iter, int flags,\n--\nkernel/trace/trace_kprobe.c-1594-{\nkernel/trace/trace_kprobe.c:1595:\tstruct kprobe_trace_entry_head *field;\nkernel/trace/trace_kprobe.c-1596-\tstruct trace_seq *s = \u0026iter-\u003eseq;\n--\nkernel/trace/trace_kprobe.c-1598-\nkernel/trace/trace_kprobe.c:1599:\tfield = (struct kprobe_trace_entry_head *)iter-\u003eent;\nkernel/trace/trace_kprobe.c-1600-\ttp = trace_probe_primary_from_call(\n--\nkernel/trace/trace_kprobe.c=1658=static int kprobe_event_define_fields(struct trace_event_call *event_call)\n--\nkernel/trace/trace_kprobe.c-1660-\tint ret;\nkernel/trace/trace_kprobe.c:1661:\tstruct kprobe_trace_entry_head field;\nkernel/trace/trace_kprobe.c-1662-\tstruct trace_probe *tp;\n--\nkernel/trace/trace_kprobe.c=1693=kprobe_perf_func(struct trace_kprobe *tk, struct pt_regs *regs)\n--\nkernel/trace/trace_kprobe.c-1695-\tstruct trace_event_call *call = trace_probe_event_call(\u0026tk-\u003etp);\nkernel/trace/trace_kprobe.c:1696:\tstruct kprobe_trace_entry_head *entry;\nkernel/trace/trace_kprobe.c-1697-\tstruct hlist_head *head;\n"
} |
| 31/3 | 2026/09/24 05:46 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 59703 output: 483 thoughts: 4284 Thoughts: **KMSAN Assessment: No Uninitialized Memory Risks** Alright, after meticulously reviewing the entire patch, I'm confident that KMSAN is not required here. The primary focus of this patch is on implementing watchpoint probes (wprobes) using hardware breakpoints within the ftrace framework, enhancing local hardware breakpoint address modification on x86, and adding KVM in-guest detection callbacks. The core concern is always whether the code introduces new risks of uninitialized memory reads or information leaks to userspace. My analysis went function by function and line by line. I checked every function involved: `local_db_save`, `local_db_restore` and functions within `arch/x86/kernel/hw_breakpoint.c` and `kernel/trace/trace_wprobe.c`. I paid close attention to where data structures are allocated, initialized, and used. The key finding is that all dynamically allocated structures, such as those related to trace probes and triggers, are consistently initialized using `kzalloc` or `kstrdup`. Any variables declared on the stack are explicitly assigned values before use. No data from uninitialized memory is exposed to user-space in any of the functions. Critically, the trace entries written to the ring buffer lack padding that could expose uninitialized data. Furthermore, data is not directly copied to user-space via `copy_to_user` or analogous mechanisms. The ftrace output is generated using a text-based format. Any security vulnerabilities would come from other problems in kernel code. Therefore, the patch does not introduce any risks of uninitialized memory usage or information leaks. Standard kernel sanitizers, particularly KASAN and LOCKDEP, are sufficient to address any potential issues. These include any risks related to concurrency, hardware breakpoint register manipulation, NULL pointer dereferences, or buffer overflows. The KMSAN flag should therefore be set to `false`. |
| 32/3 | 2026/09/24 05:46 | tool | set-results |
0mArgs:{
"KMSANReasoning": "The patch series adds support for watchpoint probe (wprobe) dynamic tracing events and event triggers based on hardware breakpoints, along with x86 architecture support for modifying local hardware breakpoint addresses (using a per-CPU sequence counter `cpu_dr7_seq` to serialize `local_db_save`/`local_db_restore`).\n\nKMSAN vs. KASAN applicability analysis:\n1. Memory initialization: All heap structures (`trace_wprobe`, `wprobe_trigger_data`, `traceprobe_parse_context`) are allocated using zeroing allocators (`kzalloc_flex`, `kzalloc_obj`), and strings use `kstrdup`.\n2. Stack variables: In `arch/x86/include/asm/debugreg.h` and callers (`exc_machine_check_*`, `exc_nmi`, `exc_debug_kernel`), the newly introduced `dr7` and `dr7_seq` output pointers are written unconditionally on the first iteration of `local_db_save()` before any read or use in `local_db_restore()`. In `arch_modify_local_hw_breakpoint_addr()` and `kernel/trace/trace_wprobe.c`, all local variables are explicitly initialized before use.\n3. User-space data exposure / padding: No structures are sent to user space via `copy_to_user`, netlink, socket options, or ioctl outputs. Trace records written to the ftrace ring buffer use `struct wprobe_trace_entry_head`, which consists of `struct trace_entry` (8 bytes) and `unsigned long ip` (8 bytes), having no structure padding. Trace output is serialized as text through the standard tracing sequencer.\n4. Failure modes: Any potential bugs in this patch (such as concurrency issues across IRQ/NMI contexts, hardware breakpoint slot tracking, refcounting/lifetime of probe objects, or string parsing bounds) are memory corruption, concurrency, or control flow issues properly covered by KASAN, LOCKDEP, and standard kernel debugging tools.\n\nTherefore, the changes do not introduce or expose uninitialized memory risks, and a dedicated KMSAN session is not warranted.",
"NeedsKMSAN": false
}Results: {
"KMSANReasoning": "The patch series adds support for watchpoint probe (wprobe) dynamic tracing events and event triggers based on hardware breakpoints, along with x86 architecture support for modifying local hardware breakpoint addresses (using a per-CPU sequence counter `cpu_dr7_seq` to serialize `local_db_save`/`local_db_restore`).\n\nKMSAN vs. KASAN applicability analysis:\n1. Memory initialization: All heap structures (`trace_wprobe`, `wprobe_trigger_data`, `traceprobe_parse_context`) are allocated using zeroing allocators (`kzalloc_flex`, `kzalloc_obj`), and strings use `kstrdup`.\n2. Stack variables: In `arch/x86/include/asm/debugreg.h` and callers (`exc_machine_check_*`, `exc_nmi`, `exc_debug_kernel`), the newly introduced `dr7` and `dr7_seq` output pointers are written unconditionally on the first iteration of `local_db_save()` before any read or use in `local_db_restore()`. In `arch_modify_local_hw_breakpoint_addr()` and `kernel/trace/trace_wprobe.c`, all local variables are explicitly initialized before use.\n3. User-space data exposure / padding: No structures are sent to user space via `copy_to_user`, netlink, socket options, or ioctl outputs. Trace records written to the ftrace ring buffer use `struct wprobe_trace_entry_head`, which consists of `struct trace_entry` (8 bytes) and `unsigned long ip` (8 bytes), having no structure padding. Trace output is serialized as text through the standard tracing sequencer.\n4. Failure modes: Any potential bugs in this patch (such as concurrency issues across IRQ/NMI contexts, hardware breakpoint slot tracking, refcounting/lifetime of probe objects, or string parsing bounds) are memory corruption, concurrency, or control flow issues properly covered by KASAN, LOCKDEP, and standard kernel debugging tools.\n\nTherefore, the changes do not introduce or expose uninitialized memory risks, and a dedicated KMSAN session is not warranted.",
"NeedsKMSAN": false
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|