From: Chuck Lever Profiling NFSD under an iozone workload showed that hardened usercopy checks consume roughly 1.3% of CPU in the TCP receive path. These checks validate memory regions during copies, but provide no security benefit when both source (skb data) and destination (kernel pages in BVEC/KVEC iterators) reside in kernel address space. Modify simple_copy_to_iter() and crc32c_and_copy_to_iter() to call _copy_to_iter() directly when the destination is a kernel-only iterator, bypassing the usercopy hardening validation. User-backed iterators (ITER_UBUF, ITER_IOVEC) continue to use copy_to_iter() with full validation. This benefits kernel consumers of TCP receive such as the NFS client and server and NVMe-TCP, which use ITER_BVEC for their receive buffers. Signed-off-by: Chuck Lever --- net/core/datagram.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/net/core/datagram.c b/net/core/datagram.c index c285c6465923..e83cf0125008 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -490,7 +490,10 @@ static size_t crc32c_and_copy_to_iter(const void *addr, size_t bytes, u32 *crcp = _crcp; size_t copied; - copied = copy_to_iter(addr, bytes, i); + if (user_backed_iter(i)) + copied = copy_to_iter(addr, bytes, i); + else + copied = _copy_to_iter(addr, bytes, i); *crcp = crc32c(*crcp, addr, copied); return copied; } @@ -515,10 +518,17 @@ int skb_copy_and_crc32c_datagram_iter(const struct sk_buff *skb, int offset, EXPORT_SYMBOL(skb_copy_and_crc32c_datagram_iter); #endif /* CONFIG_NET_CRC32C */ +/* + * Bypass usercopy hardening for kernel-only iterators: no data + * crosses the user/kernel boundary, so the slab whitelist check + * on the source buffer is unnecessary overhead. + */ static size_t simple_copy_to_iter(const void *addr, size_t bytes, void *data __always_unused, struct iov_iter *i) { - return copy_to_iter(addr, bytes, i); + if (user_backed_iter(i)) + return copy_to_iter(addr, bytes, i); + return _copy_to_iter(addr, bytes, i); } /** -- 2.52.0