| Seq | Timestamp | Type | Name | Duration |
|---|---|---|---|---|
| 0/0 | 2026/09/30 13:44 | flow | patch-triage |
10mArgs:null Results: {
"EnableConfigs": [
"NETFS_SUPPORT",
"AFS_FS",
"CIFS"
],
"FocusSymbols": [
"bvecq_alloc_one",
"bvecq_alloc_chain",
"bvecq_alloc_buffer2",
"bvecq_put",
"bvecq_expand_buffer",
"iov_iter_bvec_queue",
"rolling_buffer_init",
"rolling_buffer_make_space",
"rolling_buffer_bulk_load_from_ra",
"rolling_buffer_append",
"rolling_buffer_delete_spent",
"netfs_writeback_single",
"afs_dir_find_block",
"afs_read_dir"
],
"KMSANReasoning": "The patch series replaces the `folio_queue` mechanism with `bvecq` (segmented queue of `bio_vec` structures) across netfs, AFS, CIFS/SMB, iov_iter, and scatterlist.\n\nEvaluating against KMSAN criteria:\n1. Kernel-to-user info-leaks: No kernel structures or padded structs are exposed or copied to user space. In fact, `afs_init_new_symlink()` explicitly adds a `memset` to zero trailing page data after copying symlink content.\n2. Uninitialized memory reads / branching: `struct bvecq` is allocated in `bvecq_alloc_one()` and its fields are completely initialized using a C struct literal assignment (`*bq = (struct bvecq){...}`). In `bvecq_alloc_buffer2()`, the slot array `p-\u003ebv` is explicitly cleared with `memset()`. Slots are populated using `bvec_set_page()` or `bvec_set_folio()` (which initialize all `struct bio_vec` fields) before `nr_slots` is updated via release barrier. All iterators and consumers across netfs, smb, scatterlist, and iov_iter strictly bound accesses by `nr_slots`.\n3. KASAN vs KMSAN applicability: The risks in this patch series involve buffer boundaries, pointer arithmetic (such as reusing the tail of `bvecq-\u003ebv` for `struct page **` during bulk loads), and reference counting (`bvecq_put`/`bvecq_get`). These failure modes (out-of-bounds accesses, use-after-free, double-free) are properly detected by standard KASAN and memory debugging checks. There are no uninitialized memory reads or information leaks requiring a dedicated KMSAN session.",
"NeedsKMSAN": false,
"Reasoning": "The patch refactors the kernel's segmented folio queue (`folio_queue` / `ITER_FOLIOQ`) into a segmented bio_vec queue (`bvecq` / `ITER_BVECQ`). It introduces a new `fs/netfs/bvecq.c` implementation with custom memory management (including bulk page allocations overlaid with bio_vec structs), lockless queue manipulations using acquire/release barriers, replaces iterator iteration and extraction algorithms in `lib/iov_iter.c` and `lib/scatterlist.c`, updates netfs rolling buffers, and refactors callers in AFS and SMB/CIFS. The newly introduced and modified code runs in reachable core subsystem and filesystem paths and adds several defensive WARN_ON assertions, warranting fuzzing.",
"WorthFuzzing": true
} |
| 1/1 | 2026/09/30 13:44 | action | read-patch-diff |
0mArgs:null Results: {
"PatchDiff": "commit e4563b15c78bd5d23f58d5727c178d8ded28c31b\nAuthor: syz-cluster \u003ctriage@syzkaller.com\u003e\nDate: Wed Sep 30 13:44:48 2026 +0000\n\n syz-cluster: applied patch under review\n\ndiff --git a/Documentation/core-api/folio_queue.rst b/Documentation/core-api/folio_queue.rst\ndeleted file mode 100644\nindex b7628896d2b69..0000000000000\n--- a/Documentation/core-api/folio_queue.rst\n+++ /dev/null\n@@ -1,209 +0,0 @@\n-.. SPDX-License-Identifier: GPL-2.0+\n-\n-===========\n-Folio Queue\n-===========\n-\n-:Author: David Howells \u003cdhowells@redhat.com\u003e\n-\n-.. Contents:\n-\n- * Overview\n- * Initialisation\n- * Adding and removing folios\n- * Querying information about a folio\n- * Querying information about a folio_queue\n- * Folio queue iteration\n- * Folio marks\n- * Lockless simultaneous production/consumption issues\n-\n-\n-Overview\n-========\n-\n-The folio_queue struct forms a single segment in a segmented list of folios\n-that can be used to form an I/O buffer. As such, the list can be iterated over\n-using the ITER_FOLIOQ iov_iter type.\n-\n-The publicly accessible members of the structure are::\n-\n-\tstruct folio_queue {\n-\t\tstruct folio_queue *next;\n-\t\tstruct folio_queue *prev;\n-\t\t...\n-\t};\n-\n-A pair of pointers are provided, ``next`` and ``prev``, that point to the\n-segments on either side of the segment being accessed. Whilst this is a\n-doubly-linked list, it is intentionally not a circular list; the outward\n-sibling pointers in terminal segments should be NULL.\n-\n-Each segment in the list also stores:\n-\n- * an ordered sequence of folio pointers,\n- * the size of each folio and\n- * three 1-bit marks per folio,\n-\n-but these should not be accessed directly as the underlying data structure may\n-change, but rather the access functions outlined below should be used.\n-\n-The facility can be made accessible by::\n-\n-\t#include \u003clinux/folio_queue.h\u003e\n-\n-and to use the iterator::\n-\n-\t#include \u003clinux/uio.h\u003e\n-\n-\n-Initialisation\n-==============\n-\n-A segment should be initialised by calling::\n-\n-\tvoid folioq_init(struct folio_queue *folioq);\n-\n-with a pointer to the segment to be initialised. Note that this will not\n-necessarily initialise all the folio pointers, so care must be taken to check\n-the number of folios added.\n-\n-\n-Adding and removing folios\n-==========================\n-\n-Folios can be set in the next unused slot in a segment struct by calling one\n-of::\n-\n-\tunsigned int folioq_append(struct folio_queue *folioq,\n-\t\t\t\t struct folio *folio);\n-\n-\tunsigned int folioq_append_mark(struct folio_queue *folioq,\n-\t\t\t\t\tstruct folio *folio);\n-\n-Both functions update the stored folio count, store the folio and note its\n-size. The second function also sets the first mark for the folio added. Both\n-functions return the number of the slot used. [!] Note that no attempt is made\n-to check that the capacity wasn't overrun and the list will not be extended\n-automatically.\n-\n-A folio can be excised by calling::\n-\n-\tvoid folioq_clear(struct folio_queue *folioq, unsigned int slot);\n-\n-This clears the slot in the array and also clears all the marks for that folio,\n-but doesn't change the folio count - so future accesses of that slot must check\n-if the slot is occupied.\n-\n-\n-Querying information about a folio\n-==================================\n-\n-Information about the folio in a particular slot may be queried by the\n-following function::\n-\n-\tstruct folio *folioq_folio(const struct folio_queue *folioq,\n-\t\t\t\t unsigned int slot);\n-\n-If a folio has not yet been set in that slot, this may yield an undefined\n-pointer. The size of the folio in a slot may be queried with either of::\n-\n-\tunsigned int folioq_folio_order(const struct folio_queue *folioq,\n-\t\t\t\t\tunsigned int slot);\n-\n-\tsize_t folioq_folio_size(const struct folio_queue *folioq,\n-\t\t\t\t unsigned int slot);\n-\n-The first function returns the size as an order and the second as a number of\n-bytes.\n-\n-\n-Querying information about a folio_queue\n-========================================\n-\n-Information may be retrieved about a particular segment with the following\n-functions::\n-\n-\tunsigned int folioq_nr_slots(const struct folio_queue *folioq);\n-\n-\tunsigned int folioq_count(struct folio_queue *folioq);\n-\n-\tbool folioq_full(struct folio_queue *folioq);\n-\n-The first function returns the maximum capacity of a segment. It must not be\n-assumed that this won't vary between segments. The second returns the number\n-of folios added to a segments and the third is a shorthand to indicate if the\n-segment has been filled to capacity.\n-\n-Not that the count and fullness are not affected by clearing folios from the\n-segment. These are more about indicating how many slots in the array have been\n-initialised, and it assumed that slots won't get reused, but rather the segment\n-will get discarded as the queue is consumed.\n-\n-\n-Folio marks\n-===========\n-\n-Folios within a queue can also have marks assigned to them. These marks can be\n-used to note information such as if a folio needs folio_put() calling upon it.\n-There are three marks available to be set for each folio.\n-\n-The marks can be set by::\n-\n-\tvoid folioq_mark(struct folio_queue *folioq, unsigned int slot);\n-\tvoid folioq_mark2(struct folio_queue *folioq, unsigned int slot);\n-\n-Cleared by::\n-\n-\tvoid folioq_unmark(struct folio_queue *folioq, unsigned int slot);\n-\tvoid folioq_unmark2(struct folio_queue *folioq, unsigned int slot);\n-\n-And the marks can be queried by::\n-\n-\tbool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot);\n-\tbool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot);\n-\n-The marks can be used for any purpose and are not interpreted by this API.\n-\n-\n-Folio queue iteration\n-=====================\n-\n-A list of segments may be iterated over using the I/O iterator facility using\n-an ``iov_iter`` iterator of ``ITER_FOLIOQ`` type. The iterator may be\n-initialised with::\n-\n-\tvoid iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,\n-\t\t\t\t const struct folio_queue *folioq,\n-\t\t\t\t unsigned int first_slot, unsigned int offset,\n-\t\t\t\t size_t count);\n-\n-This may be told to start at a particular segment, slot and offset within a\n-queue. The iov iterator functions will follow the next pointers when advancing\n-and prev pointers when reverting when needed.\n-\n-\n-Lockless simultaneous production/consumption issues\n-===================================================\n-\n-If properly managed, the list can be extended by the producer at the head end\n-and shortened by the consumer at the tail end simultaneously without the need\n-to take locks. The ITER_FOLIOQ iterator inserts appropriate barriers to aid\n-with this.\n-\n-Care must be taken when simultaneously producing and consuming a list. If the\n-last segment is reached and the folios it refers to are entirely consumed by\n-the IOV iterators, an iov_iter struct will be left pointing to the last segment\n-with a slot number equal to the capacity of that segment. The iterator will\n-try to continue on from this if there's another segment available when it is\n-used again, but care must be taken lest the segment got removed and freed by\n-the consumer before the iterator was advanced.\n-\n-It is recommended that the queue always contain at least one segment, even if\n-that segment has never been filled or is entirely spent. This prevents the\n-head and tail pointers from collapsing.\n-\n-\n-API Function Reference\n-======================\n-\n-.. kernel-doc:: include/linux/folio_queue.h\ndiff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst\nindex 92f91c6a0d79d..956f6803d2a22 100644\n--- a/Documentation/core-api/index.rst\n+++ b/Documentation/core-api/index.rst\n@@ -39,7 +39,6 @@ Library functionality that is used throughout the kernel.\n kref\n cleanup\n assoc_array\n- folio_queue\n xarray\n maple_tree\n idr\ndiff --git a/Documentation/filesystems/netfs_library.rst b/Documentation/filesystems/netfs_library.rst\nindex 0c9786ffe1924..ebf07a37922e8 100644\n--- a/Documentation/filesystems/netfs_library.rst\n+++ b/Documentation/filesystems/netfs_library.rst\n@@ -452,7 +452,7 @@ be called from the writeback code to write the data to the cache, if there is\n one.\n \n The inode should be marked ``NETFS_ICTX_SINGLE_NO_UPLOAD`` if this API is to be\n-used. The writeback function requires the buffer to be of ITER_FOLIOQ type.\n+used.\n \n High-Level VM API\n ==================\ndiff --git a/fs/afs/dir.c b/fs/afs/dir.c\nindex 2db534a2c7cc4..5fac02d2d2814 100644\n--- a/fs/afs/dir.c\n+++ b/fs/afs/dir.c\n@@ -140,9 +140,9 @@ static void afs_dir_dump(struct afs_vnode *dvnode)\n \tpr_warn(\"DIR %llx:%llx is=%llx\\n\",\n \t\tdvnode-\u003efid.vid, dvnode-\u003efid.vnode, i_size);\n \n-\tiov_iter_folio_queue(\u0026iter, ITER_SOURCE, dvnode-\u003edirectory, 0, 0, i_size);\n-\titerate_folioq(\u0026iter, iov_iter_count(\u0026iter), NULL, NULL,\n-\t\t afs_dir_dump_step);\n+\tiov_iter_bvec_queue(\u0026iter, ITER_SOURCE, dvnode-\u003edirectory, 0, 0, i_size);\n+\titerate_bvecq(\u0026iter, iov_iter_count(\u0026iter), NULL, NULL,\n+\t\t afs_dir_dump_step);\n }\n \n /*\n@@ -203,9 +203,9 @@ static int afs_dir_check(struct afs_vnode *dvnode)\n \tif (unlikely(!i_size))\n \t\treturn 0;\n \n-\tiov_iter_folio_queue(\u0026iter, ITER_SOURCE, dvnode-\u003edirectory, 0, 0, i_size);\n-\tchecked = iterate_folioq(\u0026iter, iov_iter_count(\u0026iter), dvnode, NULL,\n-\t\t\t\t afs_dir_check_step);\n+\tiov_iter_bvec_queue(\u0026iter, ITER_SOURCE, dvnode-\u003edirectory, 0, 0, i_size);\n+\tchecked = iterate_bvecq(\u0026iter, iov_iter_count(\u0026iter), dvnode, NULL,\n+\t\t\t\tafs_dir_check_step);\n \tif (checked != i_size) {\n \t\tafs_dir_dump(dvnode);\n \t\treturn -EIO;\n@@ -250,15 +250,15 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)\n \tif (dvnode-\u003edirectory_size \u003c i_size) {\n \t\tsize_t cur_size = dvnode-\u003edirectory_size;\n \n-\t\tret = netfs_alloc_folioq_buffer(NULL,\n-\t\t\t\t\t\t\u0026dvnode-\u003edirectory, \u0026cur_size, i_size,\n-\t\t\t\t\t\tmapping_gfp_mask(dvnode-\u003enetfs.inode.i_mapping));\n+\t\tret = bvecq_expand_buffer(\u0026dvnode-\u003edirectory, \u0026cur_size,\n+\t\t\t\t\t round_up(i_size, PAGE_SIZE),\n+\t\t\t\t\t mapping_gfp_mask(dvnode-\u003enetfs.inode.i_mapping));\n \t\tdvnode-\u003edirectory_size = cur_size;\n \t\tif (ret \u003c 0)\n \t\t\treturn ret;\n \t}\n \n-\tiov_iter_folio_queue(\u0026iter, ITER_DEST, dvnode-\u003edirectory, 0, 0, dvnode-\u003edirectory_size);\n+\tiov_iter_bvec_queue(\u0026iter, ITER_DEST, dvnode-\u003edirectory, 0, 0, dvnode-\u003edirectory_size);\n \n \t/* AFS requires us to perform the read of a directory synchronously as\n \t * a single unit to avoid issues with the directory contents being\n@@ -294,8 +294,8 @@ static ssize_t afs_read_single(struct afs_vnode *dvnode, struct file *file)\n }\n \n /*\n- * Read the directory into a folio_queue buffer in one go, scrubbing the\n- * previous contents. We return -ESTALE if the caller needs to call us again.\n+ * Read the directory into the buffer in one go, scrubbing the previous\n+ * contents. We return -ESTALE if the caller needs to call us again.\n */\n ssize_t afs_read_dir(struct afs_vnode *dvnode, struct file *file)\n \t__acquires(\u0026dvnode-\u003evalidate_lock)\n@@ -483,7 +483,7 @@ static size_t afs_dir_iterate_step(void *iter_base, size_t progress, size_t len,\n }\n \n /*\n- * Iterate through the directory folios.\n+ * Iterate through the directory content.\n */\n static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_ctx)\n {\n@@ -498,11 +498,11 @@ static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_c\n \tif (i_size \u003c= 0 || dir_ctx-\u003epos \u003e= i_size)\n \t\treturn 0;\n \n-\tiov_iter_folio_queue(\u0026iter, ITER_SOURCE, dvnode-\u003edirectory, 0, 0, i_size);\n+\tiov_iter_bvec_queue(\u0026iter, ITER_SOURCE, dvnode-\u003edirectory, 0, 0, i_size);\n \tiov_iter_advance(\u0026iter, round_down(dir_ctx-\u003epos, AFS_DIR_BLOCK_SIZE));\n \n-\titerate_folioq(\u0026iter, iov_iter_count(\u0026iter), dvnode, \u0026ctx,\n-\t\t afs_dir_iterate_step);\n+\titerate_bvecq(\u0026iter, iov_iter_count(\u0026iter), dvnode, \u0026ctx,\n+\t\t afs_dir_iterate_step);\n \n \tif (ctx.error == -ESTALE)\n \t\tafs_invalidate_dir(dvnode, afs_dir_invalid_iter_stale);\n@@ -2228,8 +2228,8 @@ static int afs_dir_writepages(struct address_space *mapping,\n \t}\n \n \tif (test_bit(AFS_VNODE_DIR_VALID, \u0026dvnode-\u003eflags)) {\n-\t\tiov_iter_folio_queue(\u0026iter, ITER_SOURCE, dvnode-\u003edirectory, 0, 0,\n-\t\t\t\t i_size_read(\u0026dvnode-\u003enetfs.inode));\n+\t\tiov_iter_bvec_queue(\u0026iter, ITER_SOURCE, dvnode-\u003edirectory, 0, 0,\n+\t\t\t\t i_size_read(\u0026dvnode-\u003enetfs.inode));\n \t\tret = netfs_writeback_single(mapping, wbc, \u0026iter);\n \t\tif (ret == 1)\n \t\t\tret = 0; /* Skipped write due to lock conflict. */\ndiff --git a/fs/afs/dir_edit.c b/fs/afs/dir_edit.c\nindex c31303059444a..de53d7ff82c38 100644\n--- a/fs/afs/dir_edit.c\n+++ b/fs/afs/dir_edit.c\n@@ -10,7 +10,6 @@\n #include \u003clinux/namei.h\u003e\n #include \u003clinux/pagemap.h\u003e\n #include \u003clinux/iversion.h\u003e\n-#include \u003clinux/folio_queue.h\u003e\n #include \"internal.h\"\n #include \"xdr_fs.h\"\n \n@@ -110,9 +109,8 @@ static void afs_clear_contig_bits(union afs_xdr_dir_block *block,\n */\n static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, size_t block)\n {\n-\tstruct folio_queue *fq;\n \tstruct afs_vnode *dvnode = iter-\u003edvnode;\n-\tstruct folio *folio;\n+\tstruct bvecq *bq;\n \tsize_t blpos = block * AFS_DIR_BLOCK_SIZE;\n \tsize_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter-\u003efpos;\n \tint ret;\n@@ -120,41 +118,38 @@ static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, siz\n \tif (dvnode-\u003edirectory_size \u003c blend) {\n \t\tsize_t cur_size = dvnode-\u003edirectory_size;\n \n-\t\tret = netfs_alloc_folioq_buffer(\n-\t\t\tNULL, \u0026dvnode-\u003edirectory, \u0026cur_size, blend,\n-\t\t\tmapping_gfp_mask(dvnode-\u003enetfs.inode.i_mapping));\n+\t\tret = bvecq_expand_buffer(\u0026dvnode-\u003edirectory, \u0026cur_size, blend,\n+\t\t\t\t\t mapping_gfp_mask(dvnode-\u003enetfs.inode.i_mapping));\n \t\tdvnode-\u003edirectory_size = cur_size;\n \t\tif (ret \u003c 0)\n \t\t\tgoto fail;\n \t}\n \n-\tfq = iter-\u003efq;\n-\tif (!fq)\n-\t\tfq = dvnode-\u003edirectory;\n+\tbq = iter-\u003ebq;\n+\tif (!bq)\n+\t\tbq = dvnode-\u003edirectory;\n \n-\t/* Search the folio queue for the folio containing the block... */\n-\tfor (; fq; fq = fq-\u003enext) {\n-\t\tfor (int s = iter-\u003efq_slot; s \u003c folioq_count(fq); s++) {\n-\t\t\tsize_t fsize = folioq_folio_size(fq, s);\n+\t/* Search the contents for the region containing the block... */\n+\tfor (; bq; bq = bq-\u003enext) {\n+\t\tfor (int s = iter-\u003ebq_slot; s \u003c bq-\u003enr_slots; s++) {\n+\t\t\tstruct bio_vec *bv = \u0026bq-\u003ebv[s];\n+\t\t\tsize_t bsize = bv-\u003ebv_len;\n \n-\t\t\tif (blend \u003c= fpos + fsize) {\n+\t\t\tif (blend \u003c= fpos + bsize) {\n \t\t\t\t/* ... and then return the mapped block. */\n-\t\t\t\tfolio = folioq_folio(fq, s);\n-\t\t\t\tif (WARN_ON_ONCE(folio_pos(folio) != fpos))\n-\t\t\t\t\tgoto fail;\n-\t\t\t\titer-\u003efq = fq;\n-\t\t\t\titer-\u003efq_slot = s;\n+\t\t\t\titer-\u003ebq = bq;\n+\t\t\t\titer-\u003ebq_slot = s;\n \t\t\t\titer-\u003efpos = fpos;\n-\t\t\t\treturn kmap_local_folio(folio, blpos - fpos);\n+\t\t\t\treturn bvec_kmap_partial(bv, blpos - fpos);\n \t\t\t}\n-\t\t\tfpos += fsize;\n+\t\t\tfpos += bsize;\n \t\t}\n-\t\titer-\u003efq_slot = 0;\n+\t\titer-\u003ebq_slot = 0;\n \t}\n \n fail:\n-\titer-\u003efq = NULL;\n-\titer-\u003efq_slot = 0;\n+\titer-\u003ebq = NULL;\n+\titer-\u003ebq_slot = 0;\n \tafs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);\n \treturn NULL;\n }\ndiff --git a/fs/afs/dir_search.c b/fs/afs/dir_search.c\nindex 11ebdfffcb1d3..e1f91b3ac6e25 100644\n--- a/fs/afs/dir_search.c\n+++ b/fs/afs/dir_search.c\n@@ -66,12 +66,11 @@ bool afs_dir_init_iter(struct afs_dir_iter *iter, const struct qstr *name)\n */\n union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t block)\n {\n-\tstruct folio_queue *fq = iter-\u003efq;\n \tstruct afs_vnode *dvnode = iter-\u003edvnode;\n-\tstruct folio *folio;\n+\tstruct bvecq *bq = iter-\u003ebq;\n \tsize_t blpos = block * AFS_DIR_BLOCK_SIZE;\n \tsize_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter-\u003efpos;\n-\tint slot = iter-\u003efq_slot;\n+\tint slot = iter-\u003ebq_slot;\n \n \t_enter(\"%zx,%d\", block, slot);\n \n@@ -80,36 +79,34 @@ union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t bl\n \tif (dvnode-\u003edirectory_size \u003c blend)\n \t\tgoto fail;\n \n-\tif (!fq || blpos \u003c fpos) {\n-\t\tfq = dvnode-\u003edirectory;\n+\tif (!bq || blpos \u003c fpos) {\n+\t\tbq = dvnode-\u003edirectory;\n \t\tslot = 0;\n \t\tfpos = 0;\n \t}\n \n \t/* Search the folio queue for the folio containing the block... */\n-\tfor (; fq; fq = fq-\u003enext) {\n-\t\tfor (; slot \u003c folioq_count(fq); slot++) {\n-\t\t\tsize_t fsize = folioq_folio_size(fq, slot);\n+\tfor (; bq; bq = bq-\u003enext) {\n+\t\tfor (; slot \u003c bq-\u003enr_slots; slot++) {\n+\t\t\tstruct bio_vec *bv = \u0026bq-\u003ebv[slot];\n+\t\t\tsize_t bsize = bv-\u003ebv_len;\n \n-\t\t\tif (blend \u003c= fpos + fsize) {\n+\t\t\tif (blend \u003c= fpos + bsize) {\n \t\t\t\t/* ... and then return the mapped block. */\n-\t\t\t\tfolio = folioq_folio(fq, slot);\n-\t\t\t\tif (WARN_ON_ONCE(folio_pos(folio) != fpos))\n-\t\t\t\t\tgoto fail;\n-\t\t\t\titer-\u003efq = fq;\n-\t\t\t\titer-\u003efq_slot = slot;\n+\t\t\t\titer-\u003ebq = bq;\n+\t\t\t\titer-\u003ebq_slot = slot;\n \t\t\t\titer-\u003efpos = fpos;\n-\t\t\t\titer-\u003eblock = kmap_local_folio(folio, blpos - fpos);\n+\t\t\t\titer-\u003eblock = bvec_kmap_partial(bv, blpos - fpos);\n \t\t\t\treturn iter-\u003eblock;\n \t\t\t}\n-\t\t\tfpos += fsize;\n+\t\t\tfpos += bsize;\n \t\t}\n \t\tslot = 0;\n \t}\n \n fail:\n-\titer-\u003efq = NULL;\n-\titer-\u003efq_slot = 0;\n+\titer-\u003ebq = NULL;\n+\titer-\u003ebq_slot = 0;\n \tafs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);\n \treturn NULL;\n }\ndiff --git a/fs/afs/inode.c b/fs/afs/inode.c\nindex 14f39a9bea6cf..634fbf8eb2120 100644\n--- a/fs/afs/inode.c\n+++ b/fs/afs/inode.c\n@@ -683,7 +683,7 @@ void afs_evict_inode(struct inode *inode)\n \tflush_delayed_work(\u0026vnode-\u003elock_work);\n \tnetfs_wait_for_outstanding_io(inode);\n \ttruncate_inode_pages_final(\u0026inode-\u003ei_data);\n-\tnetfs_free_folioq_buffer(vnode-\u003edirectory);\n+\tbvecq_put(vnode-\u003edirectory);\n \tif (vnode-\u003esymlink)\n \t\tafs_evict_symlink(vnode);\n \ndiff --git a/fs/afs/internal.h b/fs/afs/internal.h\nindex d58273c00fbc7..51078cc9b0e55 100644\n--- a/fs/afs/internal.h\n+++ b/fs/afs/internal.h\n@@ -710,7 +710,7 @@ struct afs_vnode {\n #define AFS_VNODE_MODIFYING\t10\t\t/* Set if we're performing a modification op */\n #define AFS_VNODE_DIR_READ\t11\t\t/* Set if we've read a dir's contents */\n \n-\tstruct folio_queue\t*directory;\t/* Directory contents */\n+\tstruct bvecq\t\t*directory;\t/* Directory contents */\n \tstruct afs_symlink __rcu *symlink;\t/* Symlink content */\n \tstruct list_head\twb_keys;\t/* List of keys available for writeback */\n \tstruct list_head\tpending_locks;\t/* locks waiting to be granted */\n@@ -991,9 +991,9 @@ static inline void afs_invalidate_cache(struct afs_vnode *vnode, unsigned int fl\n struct afs_dir_iter {\n \tstruct afs_vnode\t*dvnode;\n \tunion afs_xdr_dir_block *block;\n-\tstruct folio_queue\t*fq;\n+\tstruct bvecq\t\t*bq;\n \tunsigned int\t\tfpos;\n-\tint\t\t\tfq_slot;\n+\tint\t\t\tbq_slot;\n \tunsigned int\t\tloop_check;\n \tu8\t\t\tnr_slots;\n \tu8\t\t\tbucket;\ndiff --git a/fs/afs/symlink.c b/fs/afs/symlink.c\nindex 6b8c122877ca2..9a611efe6b264 100644\n--- a/fs/afs/symlink.c\n+++ b/fs/afs/symlink.c\n@@ -56,7 +56,6 @@ void afs_evict_symlink(struct afs_vnode *vnode)\n void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)\n {\n \tstruct afs_symlink *symlink = op-\u003ecreate.symlink;\n-\tsize_t dsize = 0;\n \tsize_t size = strlen(symlink-\u003econtent) + 1;\n \tchar *p;\n \n@@ -66,13 +65,18 @@ void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)\n \tif (!fscache_cookie_enabled(netfs_i_cookie(\u0026vnode-\u003enetfs)))\n \t\treturn;\n \n-\tif (netfs_alloc_folioq_buffer(NULL, \u0026vnode-\u003edirectory, \u0026dsize, size,\n-\t\t\t\t mapping_gfp_mask(vnode-\u003enetfs.inode.i_mapping)) \u003c 0)\n+\tvnode-\u003edirectory =\n+\t\tbvecq_alloc_buffer(PAGE_SIZE,\n+\t\t\t\t mapping_gfp_mask(vnode-\u003enetfs.inode.i_mapping),\n+\t\t\t\t false);\n+\tif (!vnode-\u003edirectory)\n \t\treturn;\n \n-\tvnode-\u003edirectory_size = dsize;\n-\tp = kmap_local_folio(folioq_folio(vnode-\u003edirectory, 0), 0);\n+\tvnode-\u003edirectory_size = size;\n+\tp = bvec_kmap_partial(\u0026vnode-\u003edirectory-\u003ebv[0], 0);\n \tmemcpy(p, symlink-\u003econtent, size);\n+\tif (size \u003c PAGE_SIZE)\n+\t\tmemset(p + size, 0, PAGE_SIZE - size);\n \tkunmap_local(p);\n \tnetfs_single_mark_inode_dirty(\u0026vnode-\u003enetfs.inode);\n }\n@@ -94,17 +98,13 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)\n \t}\n \n \tif (!vnode-\u003edirectory) {\n-\t\tsize_t cur_size = 0;\n-\n-\t\tret = netfs_alloc_folioq_buffer(NULL,\n-\t\t\t\t\t\t\u0026vnode-\u003edirectory, \u0026cur_size, PAGE_SIZE,\n-\t\t\t\t\t\tmapping_gfp_mask(vnode-\u003enetfs.inode.i_mapping));\n-\t\tvnode-\u003edirectory_size = PAGE_SIZE - 1;\n-\t\tif (ret \u003c 0)\n-\t\t\treturn ret;\n+\t\tvnode-\u003edirectory = bvecq_alloc_buffer(\n+\t\t\tPAGE_SIZE, mapping_gfp_mask(vnode-\u003enetfs.inode.i_mapping), false);\n+\t\tif (!vnode-\u003edirectory)\n+\t\t\treturn -ENOMEM;\n \t}\n \n-\tiov_iter_folio_queue(\u0026iter, ITER_DEST, vnode-\u003edirectory, 0, 0, PAGE_SIZE);\n+\tiov_iter_bvec_queue(\u0026iter, ITER_DEST, vnode-\u003edirectory, 0, 0, PAGE_SIZE);\n \n \t/* AFS requires us to perform the read of a symlink as a single unit to\n \t * avoid issues with the content being changed between reads.\n@@ -126,7 +126,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)\n \t\trefcount_set(\u0026symlink-\u003eref, 1);\n \t\tsymlink-\u003econtent[i_size] = 0;\n \n-\t\tconst char *s = kmap_local_folio(folioq_folio(vnode-\u003edirectory, 0), 0);\n+\t\tconst char *s = bvec_kmap_partial(\u0026vnode-\u003edirectory-\u003ebv[0], 0);\n \n \t\tmemcpy(symlink-\u003econtent, s, i_size);\n \t\tkunmap_local(s);\n@@ -135,7 +135,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)\n \t}\n \n \tif (!fscache_cookie_enabled(netfs_i_cookie(\u0026vnode-\u003enetfs))) {\n-\t\tnetfs_free_folioq_buffer(vnode-\u003edirectory);\n+\t\tbvecq_put(vnode-\u003edirectory);\n \t\tvnode-\u003edirectory = NULL;\n \t\tvnode-\u003edirectory_size = 0;\n \t}\n@@ -248,14 +248,14 @@ int afs_symlink_writepages(struct address_space *mapping,\n \n \tif (vnode-\u003edirectory \u0026\u0026\n \t atomic64_read(\u0026vnode-\u003ecb_expires_at) != AFS_NO_CB_PROMISE) {\n-\t\tiov_iter_folio_queue(\u0026iter, ITER_SOURCE, vnode-\u003edirectory, 0, 0,\n-\t\t\t\t i_size_read(\u0026vnode-\u003enetfs.inode));\n+\t\tiov_iter_bvec_queue(\u0026iter, ITER_SOURCE, vnode-\u003edirectory, 0, 0,\n+\t\t\t\t i_size_read(\u0026vnode-\u003enetfs.inode));\n \t\tret = netfs_writeback_single(mapping, wbc, \u0026iter);\n \t}\n \n \tif (ret == 0) {\n \t\tnetfs_wb_begin(\u0026vnode-\u003enetfs, false);\n-\t\tnetfs_free_folioq_buffer(vnode-\u003edirectory);\n+\t\tbvecq_put(vnode-\u003edirectory);\n \t\tvnode-\u003edirectory = NULL;\n \t\tvnode-\u003edirectory_size = 0;\n \t\tnetfs_wb_end(\u0026vnode-\u003enetfs);\ndiff --git a/fs/netfs/Makefile b/fs/netfs/Makefile\nindex 54834cde7e568..b1ea4439c1bb4 100644\n--- a/fs/netfs/Makefile\n+++ b/fs/netfs/Makefile\n@@ -3,6 +3,7 @@\n netfs-y := \\\n \tbuffered_read.o \\\n \tbuffered_write.o \\\n+\tbvecq.o \\\n \tdirect_read.o \\\n \tdirect_write.o \\\n \titerator.o \\\ndiff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c\nindex e30bde80276a4..aa1e4f8d46ab3 100644\n--- a/fs/netfs/buffered_read.c\n+++ b/fs/netfs/buffered_read.c\n@@ -215,7 +215,7 @@ static void netfs_issue_read(struct netfs_io_request *rreq,\n * otherwise we set the deprecated PG_private_2.\n */\n static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,\n-\t\t\t\t struct folio_queue **fq,\n+\t\t\t\t struct bvecq **bq,\n \t\t\t\t unsigned int *offset,\n \t\t\t\t int *slot,\n \t\t\t\t size_t len,\n@@ -225,21 +225,21 @@ static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,\n \t\tstruct folio *folio;\n \t\tsize_t fsize, overlap;\n \n-\t\tif (!*fq)\n+\t\tif (!*bq)\n \t\t\tbreak;\n-\t\tif (*slot \u003e= folioq_count(*fq)) {\n-\t\t\t*fq = (*fq)-\u003enext;\n+\t\tif (!bvecq_acquire_slot(*bq, *slot)) {\n+\t\t\t*bq = bvecq_next(*bq);\n \t\t\t*slot = 0;\n \t\t\t*offset = 0;\n \t\t\tcontinue;\n \t\t}\n \n \t\t/* Determine how much the subreq overlaps the folio, if at all. */\n-\t\tfsize = folioq_folio_size(*fq, *slot);\n+\t\tfsize = (*bq)-\u003ebv[*slot].bv_len;\n \t\toverlap = min(len, fsize - *offset);\n \n \t\tif (overlap \u003e 0 \u0026\u0026 copy) {\n-\t\t\tfolio = folioq_folio(*fq, *slot);\n+\t\t\tfolio = bvec_folio(\u0026(*bq)-\u003ebv[*slot]);\n \t\t\tif (netfs_using_pgpriv2(rreq)) {\n \t\t\t\tif (!folio_test_private_2(folio))\n \t\t\t\t\tfolio_start_private_2(folio);\n@@ -275,7 +275,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)\n \t\t.cached_to[1]\t= ULLONG_MAX,\n \t};\n \tstruct fscache_occupancy *occ = \u0026_occ;\n-\tstruct folio_queue *fq = rreq-\u003ebuffer.tail;\n+\tstruct bvecq *bq = rreq-\u003ebuffer.tail;\n \tunsigned int offset = 0;\n \tssize_t size = rreq-\u003elen;\n \tuoff_t start = rreq-\u003estart;\n@@ -408,10 +408,10 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)\n \t\tif (size \u003c= 0)\n \t\t\tnetfs_all_subreqs_queued(rreq);\n \n-\t\tif (fq) {\n+\t\tif (bq) {\n \t\t\t/* See if the cache indicated this should be cached. */\n \t\t\tcopy = test_bit(NETFS_SREQ_COPY_TO_CACHE, \u0026subreq-\u003eflags);\n-\t\t\tnetfs_mark_copy_to_cache(rreq, \u0026fq, \u0026slot, \u0026offset, slice, copy);\n+\t\t\tnetfs_mark_copy_to_cache(rreq, \u0026bq, \u0026slot, \u0026offset, slice, copy);\n \t\t}\n \n \t\ttrace_netfs_sreq(subreq, netfs_sreq_trace_submit);\n@@ -479,8 +479,7 @@ void netfs_readahead(struct readahead_control *ractl)\n \t * acquires a ref on each folio that we will need to release later -\n \t * but we don't want to do that until after we've started the I/O.\n \t */\n-\tadded = rolling_buffer_bulk_load_from_ra(\u0026rreq-\u003ebuffer, ractl,\n-\t\t\t\t\t\t rreq-\u003edebug_id, rreq-\u003egfp);\n+\tadded = rolling_buffer_bulk_load_from_ra(\u0026rreq-\u003ebuffer, ractl, rreq-\u003egfp);\n \tif (added \u003c 0) {\n \t\tret = added;\n \t\tgoto cleanup_free;\n@@ -503,15 +502,14 @@ EXPORT_SYMBOL(netfs_readahead);\n /*\n * Create a rolling buffer with a single occupying folio.\n */\n-static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio,\n-\t\t\t\t\tunsigned int rollbuf_flags)\n+static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio)\n {\n \tssize_t added;\n \n-\tif (rolling_buffer_init(\u0026rreq-\u003ebuffer, rreq-\u003edebug_id, ITER_DEST, rreq-\u003egfp) \u003c 0)\n+\tif (rolling_buffer_init(\u0026rreq-\u003ebuffer, ITER_DEST, rreq-\u003egfp, false) \u003c 0)\n \t\treturn -ENOMEM;\n \n-\tadded = rolling_buffer_append(\u0026rreq-\u003ebuffer, folio, rollbuf_flags, rreq-\u003egfp);\n+\tadded = rolling_buffer_append(\u0026rreq-\u003ebuffer, folio, rreq-\u003egfp);\n \tif (added \u003c 0)\n \t\treturn added;\n \trreq-\u003esubmitted = rreq-\u003estart + added;\n@@ -661,7 +659,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)\n \ttrace_netfs_read(rreq, rreq-\u003estart, rreq-\u003elen, netfs_read_trace_readpage);\n \n \t/* Set up the output buffer */\n-\tret = netfs_create_singular_buffer(rreq, folio, 0);\n+\tret = netfs_create_singular_buffer(rreq, folio);\n \tif (ret \u003c 0)\n \t\tgoto discard;\n \n@@ -818,7 +816,7 @@ int netfs_write_begin(struct netfs_inode *ctx,\n \ttrace_netfs_read(rreq, pos, len, netfs_read_trace_write_begin);\n \n \t/* Set up the output buffer */\n-\tret = netfs_create_singular_buffer(rreq, folio, 0);\n+\tret = netfs_create_singular_buffer(rreq, folio);\n \tif (ret \u003c 0)\n \t\tgoto error_put;\n \n@@ -883,7 +881,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,\n \ttrace_netfs_read(rreq, start, flen, netfs_read_trace_prefetch_for_write);\n \n \t/* Set up the output buffer */\n-\tret = netfs_create_singular_buffer(rreq, folio, NETFS_ROLLBUF_PAGECACHE_MARK);\n+\tret = netfs_create_singular_buffer(rreq, folio);\n \tif (ret \u003c 0)\n \t\tgoto error_put;\n \ndiff --git a/fs/netfs/bvecq.c b/fs/netfs/bvecq.c\nnew file mode 100644\nindex 0000000000000..e78882b5672b7\n--- /dev/null\n+++ b/fs/netfs/bvecq.c\n@@ -0,0 +1,346 @@\n+// SPDX-License-Identifier: GPL-2.0-only\n+/* Buffering helpers for bvec queues\n+ *\n+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.\n+ * Written by David Howells (dhowells@redhat.com)\n+ */\n+\n+#include \u003clinux/bvecq.h\u003e\n+#include \"internal.h\"\n+\n+void bvecq_dump(const struct bvecq *bq)\n+{\n+\tint b = 0;\n+\n+\tfor (; bq; bq = bvecq_next(bq), b++) {\n+\t\tint skipz = 0;\n+\n+\t\tpr_notice(\"BQ[%u] %u/%u\\n\", b, bq-\u003enr_slots, bq-\u003emax_slots);\n+\t\tfor (int s = 0; s \u003c bq-\u003enr_slots; s++) {\n+\t\t\tconst struct bio_vec *bv = \u0026bq-\u003ebv[s];\n+\n+\t\t\tif (!bv-\u003ebv_page \u0026\u0026 !bv-\u003ebv_len \u0026\u0026 skipz \u003c 2) {\n+\t\t\t\tskipz = 1;\n+\t\t\t\tcontinue;\n+\t\t\t}\n+\t\t\tif (skipz == 1)\n+\t\t\t\tpr_notice(\"BQ[%u:00-%02u] ...\\n\", b, s - 1);\n+\t\t\tskipz = 2;\n+\t\t\tpr_notice(\"BQ[%u:%02u] %10lx %04x %04x %u\\n\",\n+\t\t\t\t b, s,\n+\t\t\t\t bv-\u003ebv_page ? page_to_pfn(bv-\u003ebv_page) : 0,\n+\t\t\t\t bv-\u003ebv_offset, bv-\u003ebv_len,\n+\t\t\t\t bv-\u003ebv_page ? page_count(bv-\u003ebv_page) : 0);\n+\t\t}\n+\t}\n+}\n+EXPORT_SYMBOL(bvecq_dump);\n+\n+/**\n+ * bvecq_alloc_one - Allocate a single bvecq node with unpopulated slots\n+ * @nr_slots: Number of slots to allocate\n+ * @gfp: The allocation constraints.\n+ * @for_writeback: True if allocating for writeback\n+ *\n+ * Allocate a single bvecq node and initialise the header. The allocation is\n+ * rounded up to the size of the smallest slab granule that will accommodate it\n+ * and all the remaining space is set as an inline slot array with max_slots\n+ * set to the number of slots. The slot array is not initialised.\n+ *\n+ * If @for_writeback is set, then the emergency mempool may be used for\n+ * allocation. If it does allocate from that pool, the maximum number of slots\n+ * available will be BVECQ_POOL_SLOTS which may be less than requested. Also,\n+ * if @for_writeback is set, the function will not fail - though it may have to\n+ * wait.\n+ *\n+ * Return: The node pointer or NULL on allocation failure.\n+ */\n+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback)\n+{\n+\tstruct bvecq *bq;\n+\tconst size_t pool_size = struct_size_t(struct bvecq, __bv, BVECQ_POOL_SLOTS);\n+\tsize_t size;\n+\tbool from_pool = false;\n+\n+\tsize = kmalloc_size_roundup(struct_size(bq, __bv, nr_slots));\n+\tgfp \u0026= ~(GFP_ZONEMASK | __GFP_THISNODE);\n+\n+\tif (for_writeback) {\n+\t\tif (size != pool_size) {\n+\t\t\tgfp_t gfp_temp = gfp;\n+\n+\t\t\tgfp_temp |= __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN;\n+\t\t\tgfp_temp \u0026= ~(__GFP_DIRECT_RECLAIM | __GFP_IO);\n+\t\t\tbq = kmalloc(size, gfp_temp);\n+\t\t\tif (bq)\n+\t\t\t\tgoto success;\n+\t\t}\n+\n+\t\tbq = mempool_alloc(\u0026netfs_bvecq_pool, gfp);\n+\t\tif (!bq)\n+\t\t\treturn bq;\n+\t\tfrom_pool = true;\n+\t\tsize = pool_size;\n+\t} else {\n+\t\tbq = kmalloc(size, gfp);\n+\t\tif (!bq)\n+\t\t\treturn bq;\n+\t}\n+\n+success:\n+\t*bq = (struct bvecq) {\n+\t\t.ref\t\t= REFCOUNT_INIT(1),\n+\t\t.bv\t\t= bq-\u003e__bv,\n+\t\t.inline_bv\t= true,\n+\t\t.max_slots\t= (size - sizeof(*bq)) / sizeof(bq-\u003e__bv[0]),\n+\t\t.from_pool\t= from_pool,\n+\t};\n+\tnetfs_stat(\u0026netfs_n_bvecq);\n+\treturn bq;\n+}\n+EXPORT_SYMBOL(bvecq_alloc_one);\n+\n+/**\n+ * bvecq_alloc_chain - Allocate an unpopulated bvecq chain\n+ * @nr_slots: Number of slots to allocate\n+ * @gfp: The allocation constraints.\n+ * @for_writeback: True if allocating for writeback\n+ *\n+ * Allocate a chain of bvecq nodes providing at least the requested cumulative\n+ * number of slots. Each node is a maximum of 4KiB in size.\n+ *\n+ * Return: The first node pointer or NULL on allocation failure.\n+ */\n+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp, bool for_writeback)\n+{\n+\tstruct bvecq *head = NULL, *tail = NULL;\n+\n+\t_enter(\"%zu\", nr_slots);\n+\n+\tfor (;;) {\n+\t\tstruct bvecq *bq;\n+\n+\t\tbq = bvecq_alloc_one(min(nr_slots, BVECQ_4KB_SLOTS), gfp, for_writeback);\n+\t\tif (!bq)\n+\t\t\tgoto oom;\n+\n+\t\tif (tail)\n+\t\t\tbvecq_append(tail, bq);\n+\t\telse\n+\t\t\thead = bq;\n+\t\ttail = bq;\n+\t\tif (tail-\u003emax_slots \u003e= nr_slots)\n+\t\t\tbreak;\n+\t\tnr_slots -= tail-\u003emax_slots;\n+\t}\n+\n+\treturn head;\n+oom:\n+\tbvecq_put(head);\n+\treturn NULL;\n+}\n+EXPORT_SYMBOL(bvecq_alloc_chain);\n+\n+/**\n+ * bvecq_alloc_buffer2 - Allocate a bvecq chain and populate with buffers\n+ * @size: Target size of the buffer (can be 0 for an empty buffer)\n+ * @pre_slots: Number of preamble slots to set aside\n+ * @gfp: The allocation constraints.\n+ * @for_writeback: True if allocating for writeback\n+ *\n+ * Allocate a chain of bvecq nodes and populate the slots with sufficient pages\n+ * to provide at least the requested amount of space, leaving the first\n+ * @pre_slots slots unset. The pre-slots must all fit into the the first\n+ * bvecq.\n+ *\n+ * The pages allocated may be compound pages larger than PAGE_SIZE and thus\n+ * occupy fewer slots. The pages have their refcounts set to 1 and can be\n+ * passed to MSG_SPLICE_PAGES.\n+ *\n+ * Return: The first node pointer or NULL on allocation failure.\n+ */\n+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp,\n+\t\t\t\t bool for_writeback)\n+{\n+\tstruct bvecq *head = NULL, *p = NULL;\n+\tsize_t nr_per_bq = BVECQ_POOL_SLOTS;\n+\tsize_t count = pre_slots + DIV_ROUND_UP(size, PAGE_SIZE);\n+\n+\t_enter(\"%zx,%zx,%u\", size, count, pre_slots);\n+\n+\tif (WARN_ON_ONCE(pre_slots \u003e nr_per_bq))\n+\t\treturn NULL;\n+\n+\thead = bvecq_alloc_chain(count, gfp, for_writeback);\n+\tif (!head)\n+\t\treturn NULL;\n+\n+\tp = head;\n+\tdo {\n+\t\tstruct page **pages;\n+\t\tsize_t unused, want, got, slot;\n+\n+\t\tif (!count)\n+\t\t\tbreak;\n+\t\tif (WARN_ON_ONCE(!p))\n+\t\t\tgoto oom;\n+\n+\t\tif (p-\u003enr_slots == 0) {\n+\t\t\t/* Need to clear pre slots and pages[], so just clear all. */\n+\t\t\tmemset(p-\u003ebv, 0, p-\u003emax_slots * sizeof(p-\u003ebv[0]));\n+\t\t\tp-\u003emem_type = BVECQ_MEM_ALLOCED;\n+\t\t\tp-\u003enr_slots = pre_slots;\n+\t\t\tcount -= pre_slots;\n+\t\t\tpre_slots = 0;\n+\t\t\tif (!count)\n+\t\t\t\tbreak;\n+\t\t}\n+\n+\t\tif (p-\u003enr_slots \u003e= p-\u003emax_slots) {\n+\t\t\tp = p-\u003enext;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tunused = p-\u003emax_slots - p-\u003enr_slots;\n+\n+\t\tpages = (struct page **)\u0026p-\u003ebv[p-\u003emax_slots];\n+\t\tpages -= unused;\n+\n+\t\twant = min(count, unused);\n+\t\tgot = alloc_pages_bulk(gfp, want, pages);\n+\t\tif (!got)\n+\t\t\tgoto oom;\n+\n+\t\tslot = p-\u003enr_slots;\n+\t\tfor (int i = 0; i \u003c got; i++) {\n+\t\t\tset_page_count(pages[i], 1);\n+\t\t\tbvec_set_page(\u0026p-\u003ebv[slot++], pages[i], PAGE_SIZE, 0);\n+\t\t}\n+\n+\t\tbvecq_filled_to(p, slot);\n+\t\tcount -= got;\n+\t} while (count \u003e 0);\n+\n+\treturn head;\n+oom:\n+\tbvecq_put(head);\n+\treturn NULL;\n+}\n+EXPORT_SYMBOL(bvecq_alloc_buffer2);\n+\n+/*\n+ * Free the page pointed to by a slot as necessary.\n+ */\n+static void bvecq_free_slot(struct bvecq *bq, unsigned int slot)\n+{\n+\tstruct page *page = bq-\u003ebv[slot].bv_page;\n+\n+\tif (!page)\n+\t\treturn;\n+\n+\tswitch (bq-\u003emem_type) {\n+\tcase BVECQ_MEM_EXTERNAL:\n+\t\tbreak;\n+\tcase BVECQ_MEM_PAGECACHE:\n+\t\tput_page(page);\n+\t\tbreak;\n+\tcase BVECQ_MEM_GUP:\n+\t\tunpin_user_page(page);\n+\t\tbreak;\n+\tcase BVECQ_MEM_ALLOCED:\n+\t\t__free_pages(page, compound_order(page));\n+\t\tbreak;\n+\tdefault:\n+\t\tWARN_ON_ONCE(1);\n+\t\tbreak;\n+\t}\n+}\n+\n+/**\n+ * bvecq_put - Put a ref on a bvec queue\n+ * @bq: The start of the folio queue to free\n+ *\n+ * Put the ref(s) on the nodes in a bvec queue, freeing up the node and the\n+ * page fragments it points to as the refcounts become zero.\n+ */\n+void bvecq_put(struct bvecq *bq)\n+{\n+\tstruct bvecq *next;\n+\n+\tfor (; bq; bq = next) {\n+\t\tif (!refcount_dec_and_test(\u0026bq-\u003eref))\n+\t\t\tbreak;\n+\t\tfor (int slot = 0; slot \u003c bq-\u003enr_slots; slot++)\n+\t\t\tbvecq_free_slot(bq, slot);\n+\t\tnext = bq-\u003enext;\n+\t\tnetfs_stat_d(\u0026netfs_n_bvecq);\n+\t\tif (bq-\u003efrom_pool)\n+\t\t\tmempool_free(bq, \u0026netfs_bvecq_pool);\n+\t\telse\n+\t\t\tkfree(bq);\n+\t}\n+}\n+EXPORT_SYMBOL(bvecq_put);\n+\n+/**\n+ * bvecq_expand_buffer - Allocate buffer space into a bvec queue\n+ * @_buffer: Pointer to the bvecq chain to expand (may point to a NULL; updated).\n+ * @_cur_size: Current size of the buffer (updated).\n+ * @size: Target size of the buffer.\n+ * @gfp: The allocation constraints.\n+ *\n+ * Append extra pages to a buffer to increase its capacity to the @size\n+ * specified. If the current tail has space, but is not of the\n+ * BVECQ_MEM_ALLOCED memory type, a separate bvecq will be allocated to hold\n+ * the new memory.\n+ */\n+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp)\n+{\n+\tstruct bvecq *tail = *_buffer;\n+\n+\tsize = round_up(size, PAGE_SIZE);\n+\tif (tail)\n+\t\twhile (tail-\u003enext)\n+\t\t\ttail = tail-\u003enext;\n+\n+\twhile (*_cur_size \u003c size) {\n+\t\tstruct page *page;\n+\t\tsize_t need = size - *_cur_size;\n+\t\tgfp_t gfp_add;\n+\t\tint order = 0;\n+\n+\t\tif (!tail || bvecq_is_full(tail) || tail-\u003emem_type != BVECQ_MEM_ALLOCED) {\n+\t\t\tstruct bvecq *p;\n+\n+\t\t\tp = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, false);\n+\t\t\tif (!p)\n+\t\t\t\treturn -ENOMEM;\n+\t\t\tif (tail)\n+\t\t\t\tbvecq_append(tail, p);\n+\t\t\telse\n+\t\t\t\t*_buffer = p;\n+\t\t\ttail = p;\n+\t\t\tp-\u003emem_type = BVECQ_MEM_ALLOCED;\n+\t\t}\n+\n+\t\tif (need \u003e PAGE_SIZE)\n+\t\t\torder = umin(ilog2(need) - PAGE_SHIFT, MAX_PAGECACHE_ORDER);\n+\n+\t\tgfp_add = 0;\n+\t\tif (order \u003e 0)\n+\t\t\tgfp_add |= __GFP_NORETRY | __GFP_NOWARN;\n+\t\tpage = alloc_pages(gfp | __GFP_COMP | gfp_add, order);\n+\t\tif (!page \u0026\u0026 order \u003e 0) {\n+\t\t\tpage = alloc_pages(gfp | __GFP_COMP, 0);\n+\t\t\torder = 0;\n+\t\t}\n+\t\tif (!page)\n+\t\t\treturn -ENOMEM;\n+\n+\t\tbvec_set_page(\u0026tail-\u003ebv[tail-\u003enr_slots], page, PAGE_SIZE \u003c\u003c order, 0);\n+\t\t*_cur_size += PAGE_SIZE \u003c\u003c order;\n+\t\tbvecq_filled_to(tail, tail-\u003enr_slots + 1);\n+\t}\n+\n+\treturn 0;\n+}\n+EXPORT_SYMBOL(bvecq_expand_buffer);\ndiff --git a/fs/netfs/internal.h b/fs/netfs/internal.h\nindex b8591abc90a98..583ffa967d5a9 100644\n--- a/fs/netfs/internal.h\n+++ b/fs/netfs/internal.h\n@@ -7,7 +7,6 @@\n \n #include \u003clinux/slab.h\u003e\n #include \u003clinux/seq_file.h\u003e\n-#include \u003clinux/folio_queue.h\u003e\n #include \u003clinux/netfs.h\u003e\n #include \u003clinux/fscache.h\u003e\n #include \u003clinux/fscache-cache.h\u003e\n@@ -45,7 +44,7 @@ extern struct list_head netfs_io_requests;\n extern spinlock_t netfs_proc_lock;\n extern mempool_t netfs_request_pool;\n extern mempool_t netfs_subrequest_pool;\n-extern mempool_t netfs_folioq_pool;\n+extern mempool_t netfs_bvecq_pool;\n \n #ifdef CONFIG_PROC_FS\n static inline void netfs_proc_add_rreq(struct netfs_io_request *rreq)\n@@ -70,8 +69,6 @@ static inline void netfs_proc_del_rreq(struct netfs_io_request *rreq) {}\n /*\n * misc.c\n */\n-struct folio_queue *netfs_buffer_make_space(struct netfs_io_request *rreq,\n-\t\t\t\t\t enum netfs_folioq_trace trace);\n void netfs_reset_iter(struct netfs_io_subrequest *subreq);\n void netfs_wake_collector(struct netfs_io_request *rreq);\n void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq);\n@@ -202,7 +199,7 @@ extern atomic_t netfs_n_wh_retry_write_req;\n extern atomic_t netfs_n_wh_retry_write_subreq;\n extern atomic_t netfs_n_wb_lock_skip;\n extern atomic_t netfs_n_wb_lock_wait;\n-extern atomic_t netfs_n_folioq;\n+extern atomic_t netfs_n_bvecq;\n \n int netfs_stats_show(struct seq_file *m, void *v);\n \ndiff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c\nindex eb1efb17f53aa..31748526d5682 100644\n--- a/fs/netfs/iterator.c\n+++ b/fs/netfs/iterator.c\n@@ -245,33 +245,34 @@ static size_t netfs_limit_xarray(const struct iov_iter *iter, size_t start_offse\n }\n \n /*\n- * Select the span of a folio queue iterator we're going to use. Limit it by\n- * both maximum size and maximum number of segments. Returns the size of the\n- * span in bytes.\n+ * Select the span of a bvecq iterator we're going to use. Limit it by both\n+ * maximum size and maximum number of segments. Returns the size of the span\n+ * in bytes.\n */\n-static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offset,\n-\t\t\t\t size_t max_size, size_t max_segs)\n+static size_t netfs_limit_bvecq(const struct iov_iter *iter, size_t start_offset,\n+\t\t\t\tsize_t max_size, size_t max_segs)\n {\n-\tconst struct folio_queue *folioq = iter-\u003efolioq;\n+\tconst struct bvecq *bq = iter-\u003ebvecq;\n \tunsigned int nsegs = 0;\n-\tunsigned int slot = iter-\u003efolioq_slot;\n+\tunsigned int slot = iter-\u003ebvecq_slot;\n \tsize_t span = 0, n = iter-\u003ecount;\n \n-\tif (WARN_ON(!iov_iter_is_folioq(iter)) ||\n+\tif (WARN_ON(!iov_iter_is_bvecq(iter)) ||\n \t WARN_ON(start_offset \u003e n) ||\n \t n == 0)\n \t\treturn 0;\n \tmax_size = umin(max_size, n - start_offset);\n \n-\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\tfolioq = folioq-\u003enext;\n+\tif (!bvecq_acquire_slot(bq, slot)) {\n+\t\tbq = bvecq_next(bq);\n \t\tslot = 0;\n \t}\n \n \tstart_offset += iter-\u003eiov_offset;\n \tdo {\n-\t\tsize_t flen = folioq_folio_size(folioq, slot);\n+\t\tsize_t flen;\n \n+\t\tflen = bq-\u003ebv[slot].bv_len;\n \t\tif (start_offset \u003c flen) {\n \t\t\tspan += flen - start_offset;\n \t\t\tnsegs++;\n@@ -283,11 +284,11 @@ static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offse\n \t\t\tbreak;\n \n \t\tslot++;\n-\t\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\t\tfolioq = folioq-\u003enext;\n+\t\tif (!bvecq_acquire_slot(bq, slot)) {\n+\t\t\tbq = bvecq_next(bq);\n \t\t\tslot = 0;\n \t\t}\n-\t} while (folioq);\n+\t} while (bq);\n \n \treturn umin(span, max_size);\n }\n@@ -295,8 +296,8 @@ static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offse\n size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,\n \t\t\tsize_t max_size, size_t max_segs)\n {\n-\tif (iov_iter_is_folioq(iter))\n-\t\treturn netfs_limit_folioq(iter, start_offset, max_size, max_segs);\n+\tif (iov_iter_is_bvecq(iter))\n+\t\treturn netfs_limit_bvecq(iter, start_offset, max_size, max_segs);\n \tif (iov_iter_is_bvec(iter))\n \t\treturn netfs_limit_bvec(iter, start_offset, max_size, max_segs);\n \tif (iov_iter_is_xarray(iter))\ndiff --git a/fs/netfs/main.c b/fs/netfs/main.c\nindex 609e22e8f76a0..36a1d98e593a0 100644\n--- a/fs/netfs/main.c\n+++ b/fs/netfs/main.c\n@@ -28,7 +28,7 @@ static struct kmem_cache *netfs_request_slab;\n static struct kmem_cache *netfs_subrequest_slab;\n mempool_t netfs_request_pool;\n mempool_t netfs_subrequest_pool;\n-mempool_t netfs_folioq_pool;\n+mempool_t netfs_bvecq_pool;\n \n #ifdef CONFIG_PROC_FS\n LIST_HEAD(netfs_io_requests);\n@@ -108,8 +108,9 @@ static int __init netfs_init(void)\n {\n \tint ret = -ENOMEM;\n \n-\tif (mempool_init_kmalloc_pool(\u0026netfs_folioq_pool, 100, sizeof(struct folio_queue)) \u003c 0)\n-\t\tgoto error_folioq_pool;\n+\tif (mempool_init_kmalloc_pool(\u0026netfs_bvecq_pool, 100,\n+\t\t\t\t struct_size_t(struct bvecq, __bv, BVECQ_POOL_SLOTS)) \u003c 0)\n+\t\tgoto error_bvecq_pool;\n \n \tnetfs_request_slab = kmem_cache_create(\"netfs_request\",\n \t\t\t\t\t sizeof(struct netfs_io_request), 0,\n@@ -163,8 +164,8 @@ static int __init netfs_init(void)\n error_reqpool:\n \tkmem_cache_destroy(netfs_request_slab);\n error_req:\n-\tmempool_exit(\u0026netfs_folioq_pool);\n-error_folioq_pool:\n+\tmempool_exit(\u0026netfs_bvecq_pool);\n+error_bvecq_pool:\n \treturn ret;\n }\n fs_initcall(netfs_init);\n@@ -177,6 +178,6 @@ static void __exit netfs_exit(void)\n \tkmem_cache_destroy(netfs_subrequest_slab);\n \tmempool_exit(\u0026netfs_request_pool);\n \tkmem_cache_destroy(netfs_request_slab);\n-\tmempool_exit(\u0026netfs_folioq_pool);\n+\tmempool_exit(\u0026netfs_bvecq_pool);\n }\n module_exit(netfs_exit);\ndiff --git a/fs/netfs/misc.c b/fs/netfs/misc.c\nindex a3cd76d584b8b..40e0ff6491329 100644\n--- a/fs/netfs/misc.c\n+++ b/fs/netfs/misc.c\n@@ -8,102 +8,6 @@\n #include \u003clinux/swap.h\u003e\n #include \"internal.h\"\n \n-/**\n- * netfs_alloc_folioq_buffer - Allocate buffer space into a folio queue\n- * @mapping: Address space to set on the folio (or NULL).\n- * @_buffer: Pointer to the folio queue to add to (may point to a NULL; updated).\n- * @_cur_size: Current size of the buffer (updated).\n- * @size: Target size of the buffer.\n- * @gfp: The allocation constraints.\n- */\n-int netfs_alloc_folioq_buffer(struct address_space *mapping,\n-\t\t\t struct folio_queue **_buffer,\n-\t\t\t size_t *_cur_size, ssize_t size, gfp_t gfp)\n-{\n-\tstruct folio_queue *tail = *_buffer, *p;\n-\n-\tsize = round_up(size, PAGE_SIZE);\n-\tif (*_cur_size \u003e= size)\n-\t\treturn 0;\n-\n-\tif (tail)\n-\t\twhile (tail-\u003enext)\n-\t\t\ttail = tail-\u003enext;\n-\n-\tdo {\n-\t\tstruct folio *folio;\n-\t\tint order = 0, slot;\n-\n-\t\tif (!tail || folioq_full(tail)) {\n-\t\t\tp = netfs_folioq_alloc(0, GFP_NOFS, netfs_trace_folioq_alloc_buffer);\n-\t\t\tif (!p)\n-\t\t\t\treturn -ENOMEM;\n-\t\t\tif (tail) {\n-\t\t\t\ttail-\u003enext = p;\n-\t\t\t\tp-\u003eprev = tail;\n-\t\t\t} else {\n-\t\t\t\t*_buffer = p;\n-\t\t\t}\n-\t\t\ttail = p;\n-\t\t}\n-\n-\t\tif (size - *_cur_size \u003e PAGE_SIZE)\n-\t\t\torder = umin(ilog2(size - *_cur_size) - PAGE_SHIFT,\n-\t\t\t\t MAX_PAGECACHE_ORDER);\n-\n-\t\tfolio = folio_alloc(gfp, order);\n-\t\tif (!folio \u0026\u0026 order \u003e 0)\n-\t\t\tfolio = folio_alloc(gfp, 0);\n-\t\tif (!folio)\n-\t\t\treturn -ENOMEM;\n-\n-\t\tfolio-\u003emapping = mapping;\n-\t\tfolio-\u003eindex = *_cur_size / PAGE_SIZE;\n-\t\ttrace_netfs_folio(folio, netfs_folio_trace_alloc_buffer);\n-\t\tslot = folioq_append_mark(tail, folio);\n-\t\t*_cur_size += folioq_folio_size(tail, slot);\n-\t} while (*_cur_size \u003c size);\n-\n-\treturn 0;\n-}\n-EXPORT_SYMBOL(netfs_alloc_folioq_buffer);\n-\n-/**\n- * netfs_free_folioq_buffer - Free a folio queue.\n- * @fq: The start of the folio queue to free\n- *\n- * Free up a chain of folio_queues and, if marked, the marked folios they point\n- * to.\n- */\n-void netfs_free_folioq_buffer(struct folio_queue *fq)\n-{\n-\tstruct folio_queue *next;\n-\tstruct folio_batch fbatch;\n-\n-\tfolio_batch_init(\u0026fbatch);\n-\n-\tfor (; fq; fq = next) {\n-\t\tfor (int slot = 0; slot \u003c folioq_count(fq); slot++) {\n-\t\t\tstruct folio *folio = folioq_folio(fq, slot);\n-\n-\t\t\tif (!folio ||\n-\t\t\t !folioq_is_marked(fq, slot))\n-\t\t\t\tcontinue;\n-\n-\t\t\ttrace_netfs_folio(folio, netfs_folio_trace_put);\n-\t\t\tif (folio_batch_add(\u0026fbatch, folio))\n-\t\t\t\tfolio_batch_release(\u0026fbatch);\n-\t\t}\n-\n-\t\tnetfs_stat_d(\u0026netfs_n_folioq);\n-\t\tnext = fq-\u003enext;\n-\t\tkfree(fq);\n-\t}\n-\n-\tfolio_batch_release(\u0026fbatch);\n-}\n-EXPORT_SYMBOL(netfs_free_folioq_buffer);\n-\n /*\n * Reset the subrequest iterator to refer just to the region remaining to be\n * read. The iterator may or may not have been advanced by socket ops or\ndiff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c\nindex 2625efd48a9b1..6bbaaea693543 100644\n--- a/fs/netfs/read_collect.c\n+++ b/fs/netfs/read_collect.c\n@@ -63,11 +63,11 @@ void netfs_cancel_copy_to_cache(struct netfs_io_request *rreq, struct folio *fol\n * dirty and let writeback handle it.\n */\n static void netfs_unlock_read_folio(struct netfs_io_request *rreq,\n-\t\t\t\t struct folio_queue *folioq,\n+\t\t\t\t struct bvecq *bq,\n \t\t\t\t int slot)\n {\n \tstruct netfs_folio *finfo;\n-\tstruct folio *folio = folioq_folio(folioq, slot);\n+\tstruct folio *folio = bvec_folio(\u0026bq-\u003ebv[slot]);\n \n \tif (unlikely(folio_pos(folio) \u003c rreq-\u003eabandon_to)) {\n \t\ttrace_netfs_folio(folio, netfs_folio_trace_abandon);\n@@ -98,7 +98,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,\n \t\t\ttrace_netfs_folio(folio, netfs_folio_trace_read_done);\n \t\t}\n \n-\t\tfolioq_clear(folioq, slot);\n+\t\tbq-\u003ebv[slot].bv_page = NULL;\n \t} else {\n \t\t// TODO: Use of PG_private_2 is deprecated.\n \t\tif (folio_test_private_2(folio))\n@@ -114,7 +114,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,\n \t\tfolio_unlock(folio);\n \t}\n \n-\tfolioq_clear(folioq, slot);\n+\tbq-\u003ebv[slot].bv_page = NULL;\n }\n \n /*\n@@ -122,21 +122,21 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,\n */\n void netfs_read_set_unlock_at(struct netfs_io_request *rreq)\n {\n-\tstruct folio_queue *folioq = rreq-\u003ebuffer.tail;\n+\tconst struct bvecq *bq = rreq-\u003ebuffer.tail;\n \tunsigned int slot = rreq-\u003ebuffer.first_tail_slot;\n \tsize_t cleaned_to = rreq-\u003ecleaned_to - rreq-\u003estart;\n \tsize_t progress_at = cleaned_to;\n \tsize_t minimum = 256 * 1024;\n \n \twhile (progress_at \u003c rreq-\u003elen) {\n-\t\tif (slot \u003e= folioq_count(folioq)) {\n-\t\t\tfolioq = folioq-\u003enext;\n-\t\t\tif (!folioq)\n+\t\tif (!bvecq_acquire_slot(bq, slot)) {\n+\t\t\tbq = bvecq_next(bq);\n+\t\t\tif (!bq)\n \t\t\t\tbreak;\n \t\t\tslot = 0;\n \t\t}\n \n-\t\tprogress_at += folioq_folio_size(folioq, slot);\n+\t\tprogress_at += bq-\u003ebv[slot].bv_len;\n \t\tif (progress_at - cleaned_to \u003e= minimum)\n \t\t\tbreak;\n \t\tslot++;\n@@ -152,7 +152,7 @@ void netfs_read_set_unlock_at(struct netfs_io_request *rreq)\n static void netfs_read_unlock_folios(struct netfs_io_request *rreq,\n \t\t\t\t unsigned int *notes)\n {\n-\tstruct folio_queue *folioq = rreq-\u003ebuffer.tail;\n+\tstruct bvecq *bq = rreq-\u003ebuffer.tail;\n \tunsigned int slot = rreq-\u003ebuffer.first_tail_slot;\n \tuoff_t collected_to = rreq-\u003ecollected_to;\n \n@@ -161,9 +161,9 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,\n \n \t// TODO: Begin decryption\n \n-\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\tfolioq = rolling_buffer_delete_spent(\u0026rreq-\u003ebuffer);\n-\t\tif (!folioq) {\n+\tif (!bvecq_acquire_slot(bq, slot)) {\n+\t\tbq = rolling_buffer_delete_spent(\u0026rreq-\u003ebuffer);\n+\t\tif (!bq) {\n \t\t\tWRITE_ONCE(rreq-\u003eprogress_at, rreq-\u003elen);\n \t\t\treturn;\n \t\t}\n@@ -182,13 +182,13 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,\n \t\tuoff_t fpos, fend;\n \t\tsize_t fsize;\n \n-\t\tfolio = folioq_folio(folioq, slot);\n+\t\tfolio = bvec_folio(\u0026bq-\u003ebv[slot]);\n \t\tif (WARN_ONCE(!folio_test_locked(folio),\n \t\t\t \"R=%08x: folio %lx is not locked\\n\",\n \t\t\t rreq-\u003edebug_id, folio-\u003eindex))\n \t\t\ttrace_netfs_folio(folio, netfs_folio_trace_not_locked);\n \n-\t\tfsize = folioq_folio_size(folioq, slot);\n+\t\tfsize = bq-\u003ebv[slot].bv_len;\n \t\tfpos = folio_pos(folio);\n \t\tfend = fpos + fsize;\n \n@@ -198,29 +198,28 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,\n \t\tif (collected_to \u003c fend)\n \t\t\tbreak;\n \n-\t\tnetfs_unlock_read_folio(rreq, folioq, slot);\n+\t\tnetfs_unlock_read_folio(rreq, bq, slot);\n \t\tWRITE_ONCE(rreq-\u003ecleaned_to, fpos + fsize);\n \t\t*notes |= MADE_PROGRESS;\n \n-\t\t/* Clean up the head folioq. If we clear an entire folioq, then\n-\t\t * we can get rid of it provided it's not also the tail folioq\n+\t\t/* Clean up the head bq. If we clear an entire bq, then\n+\t\t * we can get rid of it provided it's not also the tail bq\n \t\t * being filled by the issuer.\n \t\t */\n-\t\tfolioq_clear(folioq, slot);\n+\t\tbq-\u003ebv[slot].bv_page = NULL;\n \t\tslot++;\n-\t\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\t\tfolioq = rolling_buffer_delete_spent(\u0026rreq-\u003ebuffer);\n-\t\t\tif (!folioq)\n+\t\tif (!bvecq_acquire_slot(bq, slot)) {\n+\t\t\tbq = rolling_buffer_delete_spent(\u0026rreq-\u003ebuffer);\n+\t\t\tif (!bq)\n \t\t\t\tgoto done;\n \t\t\tslot = 0;\n-\t\t\ttrace_netfs_folioq(folioq, netfs_trace_folioq_read_progress);\n \t\t}\n \n \t\tif (fpos + fsize \u003e= collected_to)\n \t\t\tbreak;\n \t}\n \n-\trreq-\u003ebuffer.tail = folioq;\n+\trreq-\u003ebuffer.tail = bq;\n done:\n \trreq-\u003ebuffer.first_tail_slot = slot;\n \ndiff --git a/fs/netfs/read_pgpriv2.c b/fs/netfs/read_pgpriv2.c\nindex 5280b606fda41..16d623ca9df3e 100644\n--- a/fs/netfs/read_pgpriv2.c\n+++ b/fs/netfs/read_pgpriv2.c\n@@ -53,7 +53,7 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio\n \ttrace_netfs_folio(folio, netfs_folio_trace_store_copy);\n \n \t/* Attach the folio to the rolling buffer. */\n-\tif (rolling_buffer_append(\u0026creq-\u003ebuffer, folio, 0, creq-\u003egfp) \u003c 0) {\n+\tif (rolling_buffer_append(\u0026creq-\u003ebuffer, folio, creq-\u003egfp) \u003c 0) {\n \t\tset_bit(NETFS_RREQ_CANCEL_CACHING, \u0026creq-\u003eflags);\n \t\tfolio_end_private_2(folio);\n \t\treturn;\n@@ -173,13 +173,13 @@ void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)\n */\n bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)\n {\n-\tstruct folio_queue *folioq = creq-\u003ebuffer.tail;\n+\tstruct bvecq *bq = creq-\u003ebuffer.tail;\n \tunsigned int slot = creq-\u003ebuffer.first_tail_slot;\n \tuoff_t collected_to = creq-\u003ecollected_to;\n \tbool made_progress = false;\n \n-\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\tfolioq = rolling_buffer_delete_spent(\u0026creq-\u003ebuffer);\n+\tif (!bvecq_acquire_slot(bq, slot)) {\n+\t\tbq = rolling_buffer_delete_spent(\u0026creq-\u003ebuffer);\n \t\tslot = 0;\n \t}\n \n@@ -188,7 +188,7 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)\n \t\tuoff_t fpos, fend;\n \t\tsize_t fsize, flen;\n \n-\t\tfolio = folioq_folio(folioq, slot);\n+\t\tfolio = bvec_folio(\u0026bq-\u003ebv[slot]);\n \t\tif (WARN_ONCE(!folio_test_private_2(folio),\n \t\t\t \"R=%08x: folio %lx is not marked private_2\\n\",\n \t\t\t creq-\u003edebug_id, folio-\u003eindex))\n@@ -211,15 +211,15 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)\n \t\tcreq-\u003ecleaned_to = fpos + fsize;\n \t\tmade_progress = true;\n \n-\t\t/* Clean up the head folioq. If we clear an entire folioq, then\n-\t\t * we can get rid of it provided it's not also the tail folioq\n+\t\t/* Clean up the head bq. If we clear an entire bq, then\n+\t\t * we can get rid of it provided it's not also the tail bq\n \t\t * being filled by the issuer.\n \t\t */\n-\t\tfolioq_clear(folioq, slot);\n+\t\tbq-\u003ebv[slot].bv_page = NULL;\n \t\tslot++;\n-\t\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\t\tfolioq = rolling_buffer_delete_spent(\u0026creq-\u003ebuffer);\n-\t\t\tif (!folioq)\n+\t\tif (!bvecq_acquire_slot(bq, slot)) {\n+\t\t\tbq = rolling_buffer_delete_spent(\u0026creq-\u003ebuffer);\n+\t\t\tif (!bq)\n \t\t\t\tgoto done;\n \t\t\tslot = 0;\n \t\t}\n@@ -228,7 +228,7 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)\n \t\t\tbreak;\n \t}\n \n-\tcreq-\u003ebuffer.tail = folioq;\n+\tcreq-\u003ebuffer.tail = bq;\n done:\n \tcreq-\u003ebuffer.first_tail_slot = slot;\n \treturn made_progress;\ndiff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c\nindex 5bd8dee5a8346..142c3fb8dab17 100644\n--- a/fs/netfs/read_retry.c\n+++ b/fs/netfs/read_retry.c\n@@ -291,7 +291,7 @@ void netfs_retry_reads(struct netfs_io_request *rreq)\n */\n void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)\n {\n-\tstruct folio_queue *p;\n+\tstruct bvecq *p;\n \n \t/* We have to wait for readahead refs to have been released before we\n \t * can unlock any folios as the ref-dropper walks i_pages and the only\n@@ -301,24 +301,25 @@ void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)\n \t\tnetfs_wait_for_put_ra_refs(rreq);\n \n \tfor (p = rreq-\u003ebuffer.tail; p; p = p-\u003enext) {\n-\t\tfor (int slot = 0; slot \u003c folioq_count(p); slot++) {\n-\t\t\tstruct folio *folio = folioq_folio(p, slot);\n+\t\tfor (int slot = rreq-\u003ebuffer.first_tail_slot;\n+\t\t bvecq_acquire_slot(p, slot);\n+\t\t slot++) {\n+\t\t\tstruct folio *folio;\n \n-\t\t\tif (!folio)\n+\t\t\tif (!p-\u003ebv[slot].bv_page)\n \t\t\t\tcontinue;\n+\n+\t\t\tfolio = bvec_folio(\u0026p-\u003ebv[slot]);\n \t\t\tnetfs_cancel_copy_to_cache(rreq, folio);\n \n-\t\t\tif (!folioq_is_marked2(p, slot)) {\n-\t\t\t\tif (folio == rreq-\u003eno_unlock_folio \u0026\u0026\n-\t\t\t\t test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO,\n-\t\t\t\t\t \u0026rreq-\u003eflags)) {\n-\t\t\t\t\t_debug(\"no unlock\");\n-\t\t\t\t} else {\n-\t\t\t\t\ttrace_netfs_folio(folio,\n-\t\t\t\t\t\tnetfs_folio_trace_abandon);\n-\t\t\t\t\tfolio_unlock(folio);\n-\t\t\t\t}\n+\t\t\tif (folio == rreq-\u003eno_unlock_folio \u0026\u0026\n+\t\t\t test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, \u0026rreq-\u003eflags)) {\n+\t\t\t\t_debug(\"no unlock\");\n+\t\t\t} else {\n+\t\t\t\ttrace_netfs_folio(folio, netfs_folio_trace_abandon);\n+\t\t\t\tfolio_unlock(folio);\n \t\t\t}\n \t\t}\n+\t\trreq-\u003ebuffer.first_tail_slot = 0;\n \t}\n }\ndiff --git a/fs/netfs/rolling_buffer.c b/fs/netfs/rolling_buffer.c\nindex d30d5ef6d86ec..66ce9add40122 100644\n--- a/fs/netfs/rolling_buffer.c\n+++ b/fs/netfs/rolling_buffer.c\n@@ -12,96 +12,51 @@\n #include \u003clinux/slab.h\u003e\n #include \"internal.h\"\n \n-static atomic_t debug_ids;\n-\n-/**\n- * netfs_folioq_alloc - Allocate a folio_queue struct\n- * @rreq_id: Associated debugging ID for tracing purposes\n- * @gfp: Allocation constraints\n- * @trace: Trace tag to indicate the purpose of the allocation\n- *\n- * Allocate, initialise and account the folio_queue struct and log a trace line\n- * to mark the allocation.\n- */\n-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,\n-\t\t\t\t unsigned int /*enum netfs_folioq_trace*/ trace)\n-{\n-\tstruct folio_queue *fq;\n-\n-\tif (gfp == GFP_KERNEL)\n-\t\tfq = mempool_alloc_noreserve(\u0026netfs_folioq_pool, gfp);\n-\telse\n-\t\tfq = mempool_alloc(\u0026netfs_folioq_pool, gfp);\n-\tif (fq) {\n-\t\tnetfs_stat(\u0026netfs_n_folioq);\n-\t\tfolioq_init(fq, rreq_id);\n-\t\tfq-\u003edebug_id = atomic_inc_return(\u0026debug_ids);\n-\t\ttrace_netfs_folioq(fq, trace);\n-\t}\n-\treturn fq;\n-}\n-EXPORT_SYMBOL(netfs_folioq_alloc);\n-\n-/**\n- * netfs_folioq_free - Free a folio_queue struct\n- * @folioq: The object to free\n- * @trace: Trace tag to indicate which free\n- *\n- * Free and unaccount the folio_queue struct.\n- */\n-void netfs_folioq_free(struct folio_queue *folioq,\n-\t\t unsigned int /*enum netfs_trace_folioq*/ trace)\n-{\n-\ttrace_netfs_folioq(folioq, trace);\n-\tnetfs_stat_d(\u0026netfs_n_folioq);\n-\tmempool_free(folioq, \u0026netfs_folioq_pool);\n-}\n-EXPORT_SYMBOL(netfs_folioq_free);\n-\n /*\n * Initialise a rolling buffer. We allocate an empty folio queue struct to so\n * that the pointers can be independently driven by the producer and the\n * consumer.\n */\n-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,\n-\t\t\tunsigned int direction, gfp_t gfp)\n+int rolling_buffer_init(struct rolling_buffer *roll, unsigned int direction,\n+\t\t\tgfp_t gfp, bool for_writeback)\n {\n-\tstruct folio_queue *fq;\n+\tstruct bvecq *bq;\n+\n+\troll-\u003efor_writeback = for_writeback;\n \n-\tfq = netfs_folioq_alloc(rreq_id, gfp, netfs_trace_folioq_rollbuf_init);\n-\tif (!fq)\n+\tbq = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, for_writeback);\n+\tif (!bq)\n \t\treturn -ENOMEM;\n \n-\troll-\u003ehead = fq;\n-\troll-\u003etail = fq;\n-\tiov_iter_folio_queue(\u0026roll-\u003eiter, direction, fq, 0, 0, 0);\n+\troll-\u003ehead = bq;\n+\troll-\u003etail = bq;\n+\tiov_iter_bvec_queue(\u0026roll-\u003eiter, direction, bq, 0, 0, 0);\n \treturn 0;\n }\n \n /*\n- * Add another folio_queue to a rolling buffer if there's no space left.\n+ * Add another bvecq to a rolling buffer if there's no space left.\n */\n int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)\n {\n-\tstruct folio_queue *fq, *head = roll-\u003ehead;\n+\tstruct bvecq *bq, *head = roll-\u003ehead;\n \n-\tif (!folioq_full(head))\n+\tif (!bvecq_is_full(head))\n \t\treturn 0;\n \n-\tfq = netfs_folioq_alloc(head-\u003erreq_id, gfp, netfs_trace_folioq_make_space);\n-\tif (!fq)\n+\tbq = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, roll-\u003efor_writeback);\n+\tif (!bq)\n \t\treturn -ENOMEM;\n-\tfq-\u003eprev = head;\n \n-\troll-\u003ehead = fq;\n-\tif (folioq_full(head)) {\n+\troll-\u003ehead = bq;\n+\tif (bvecq_is_full(head)) {\n \t\t/* Make sure we don't leave the master iterator pointing to a\n \t\t * block that might get immediately consumed.\n \t\t */\n-\t\tif (roll-\u003eiter.folioq == head \u0026\u0026\n-\t\t roll-\u003eiter.folioq_slot == folioq_nr_slots(head)) {\n-\t\t\troll-\u003eiter.folioq = fq;\n-\t\t\troll-\u003eiter.folioq_slot = 0;\n+\t\tif (roll-\u003eiter.bvecq == head \u0026\u0026\n+\t\t roll-\u003eiter.bvecq_slot == head-\u003enr_slots) {\n+\t\t\troll-\u003eiter.bvecq = bq;\n+\t\t\troll-\u003eiter.bvecq_slot = 0;\n \t\t}\n \t}\n \n@@ -110,7 +65,7 @@ int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)\n \t * [!] NOTE: After we set head-\u003enext, the consumer is at liberty to\n \t * immediately delete the old head.\n \t */\n-\tsmp_store_release(\u0026head-\u003enext, fq);\n+\tbvecq_append(head, bq);\n \treturn 0;\n }\n \n@@ -119,55 +74,59 @@ int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)\n */\n ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,\n \t\t\t\t\t struct readahead_control *ractl,\n-\t\t\t\t\t unsigned int rreq_id, gfp_t gfp)\n+\t\t\t\t\t gfp_t gfp)\n {\n-\tstruct folio_queue *fq;\n-\tssize_t loaded = 0;\n+\tstruct bvecq *bq;\n+\tsize_t loaded = 0;\n \n \twhile (ractl-\u003e_nr_pages - ractl-\u003e_batch_count \u003e 0) {\n+\t\tstruct page **pages;\n \t\tunsigned int nr;\n \n-\t\t/* Allocate a folioq to put some folios into and attach it to\n+\t\t/* Allocate a bvecq to put some folios into and attach it to\n \t\t * the rolling buffer.\n \t\t */\n-\t\tfq = netfs_folioq_alloc(rreq_id, gfp,\n-\t\t\t\t\tnetfs_trace_folioq_make_space);\n-\t\tif (!fq)\n+\t\tbq = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, false);\n+\t\tif (!bq)\n \t\t\tgoto nomem_unlock;\n-\t\tfq-\u003eprev = roll-\u003ehead;\n+\t\tbq-\u003emem_type = BVECQ_MEM_EXTERNAL; /* Folio cleanup handled separately. */\n+\n \t\tif (!roll-\u003etail)\n-\t\t\troll-\u003etail = fq;\n+\t\t\troll-\u003etail = bq;\n \t\telse\n-\t\t\troll-\u003ehead-\u003enext = fq;\n-\t\troll-\u003ehead = fq;\n+\t\t\tbvecq_append(roll-\u003ehead, bq);\n+\t\troll-\u003ehead = bq;\n \n-\t\t/* Get a batch of folios and note their orders. */\n-\t\tnr = __readahead_batch(ractl, (struct page **)fq-\u003evec.folios,\n-\t\t\t\t folioq_nr_slots(fq));\n+\t\t/* Get a bunch of folios and note their sizes. */\n+\t\tpages = (struct page **)(bq-\u003ebv + bq-\u003emax_slots);\n+\t\tpages -= bq-\u003emax_slots;\n+\t\tnr = __readahead_batch(ractl, pages, bq-\u003emax_slots);\n \t\tif (WARN_ON_ONCE(!nr))\n \t\t\tbreak;\n-\t\tfq-\u003evec.nr = nr;\n \n \t\tfor (int slot = 0; slot \u003c nr; slot++) {\n-\t\t\tstruct folio *folio = folioq_folio(fq, slot);\n-\t\t\tunsigned int order;\n+\t\t\tstruct folio *folio = page_folio(pages[slot]);\n+\t\t\tsize_t len = folio_size(folio);\n \n-\t\t\torder = folio_order(folio);\n-\t\t\tfq-\u003eorders[slot] = order;\n-\t\t\tloaded += PAGE_SIZE \u003c\u003c order;\n+\t\t\tbvec_set_folio(\u0026bq-\u003ebv[slot], folio, len, 0);\n+\t\t\tloaded += len;\n \t\t\ttrace_netfs_folio(folio, netfs_folio_trace_read);\n \t\t}\n+\n+\t\tbvecq_filled_to(bq, nr);\n \t}\n \n \tWRITE_ONCE(roll-\u003eiter.count, loaded);\n-\tiov_iter_folio_queue(\u0026roll-\u003eiter, ITER_DEST, roll-\u003etail, 0, 0, loaded);\n+\tiov_iter_bvec_queue(\u0026roll-\u003eiter, ITER_DEST, roll-\u003etail, 0, 0, loaded);\n \treturn loaded;\n \n nomem_unlock:\n-\tfor (fq = roll-\u003etail; fq; fq = fq-\u003enext) {\n-\t\tfor (int slot = 0; slot \u003c folioq_count(fq); slot++) {\n-\t\t\tfolio_unlock(fq-\u003evec.folios[slot]);\n-\t\t\tfolioq_mark(fq, slot);\n+\tfor (bq = roll-\u003etail; bq; bq = bq-\u003enext) {\n+\t\tfor (int slot = 0; slot \u003c bq-\u003enr_slots; slot++) {\n+\t\t\tstruct folio *folio = bvec_folio(\u0026bq-\u003ebv[slot]);\n+\n+\t\t\tfolio_unlock(folio);\n+\t\t\tfolio_put(folio);\n \t\t}\n \t}\n \trolling_buffer_clear(roll);\n@@ -180,7 +139,7 @@ ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,\n * Append a folio to the rolling buffer.\n */\n ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,\n-\t\t\t unsigned int flags, gfp_t gfp)\n+\t\t\t gfp_t gfp)\n {\n \tssize_t size = folio_size(folio);\n \tint slot;\n@@ -188,16 +147,11 @@ ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,\n \tif (rolling_buffer_make_space(roll, gfp) \u003c 0)\n \t\treturn -ENOMEM;\n \n-\tslot = folioq_append(roll-\u003ehead, folio);\n-\tif (flags \u0026 ROLLBUF_MARK_1)\n-\t\tfolioq_mark(roll-\u003ehead, slot);\n-\tif (flags \u0026 ROLLBUF_MARK_2)\n-\t\tfolioq_mark2(roll-\u003ehead, slot);\n+\tslot = roll-\u003ehead-\u003enr_slots;\n+\tbvec_set_folio(\u0026roll-\u003ehead-\u003ebv[slot], folio, size, 0);\n+\tbvecq_filled_to(roll-\u003ehead, slot + 1);\n \n \tWRITE_ONCE(roll-\u003eiter.count, roll-\u003eiter.count + size);\n-\n-\t/* Store the counter after setting the slot. */\n-\tsmp_store_release(\u0026roll-\u003enext_head_slot, slot);\n \treturn size;\n }\n \n@@ -206,44 +160,23 @@ ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,\n * don't return the last buffer to keep the pointers independent, but return\n * NULL instead.\n */\n-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll)\n+struct bvecq *rolling_buffer_delete_spent(struct rolling_buffer *roll)\n {\n-\tstruct folio_queue *spent = roll-\u003etail, *next = READ_ONCE(spent-\u003enext);\n+\tstruct bvecq *spent = roll-\u003etail, *next = bvecq_next(spent);\n \n \tif (!next)\n \t\treturn NULL;\n \tnext-\u003eprev = NULL;\n-\tnetfs_folioq_free(spent, netfs_trace_folioq_delete);\n \troll-\u003etail = next;\n+\tspent-\u003enext = NULL;\n+\tbvecq_put(spent);\n \treturn next;\n }\n \n /*\n- * Clear out a rolling queue. Folios that have mark 1 set are put.\n+ * Clear out a rolling queue.\n */\n void rolling_buffer_clear(struct rolling_buffer *roll)\n {\n-\tstruct folio_batch fbatch;\n-\tstruct folio_queue *p;\n-\n-\tfolio_batch_init(\u0026fbatch);\n-\n-\twhile ((p = roll-\u003etail)) {\n-\t\troll-\u003etail = p-\u003enext;\n-\t\tfor (int slot = 0; slot \u003c folioq_count(p); slot++) {\n-\t\t\tstruct folio *folio = folioq_folio(p, slot);\n-\n-\t\t\tif (!folio)\n-\t\t\t\tcontinue;\n-\t\t\tif (folioq_is_marked(p, slot)) {\n-\t\t\t\ttrace_netfs_folio(folio, netfs_folio_trace_put);\n-\t\t\t\tif (!folio_batch_add(\u0026fbatch, folio))\n-\t\t\t\t\tfolio_batch_release(\u0026fbatch);\n-\t\t\t}\n-\t\t}\n-\n-\t\tnetfs_folioq_free(p, netfs_trace_folioq_clear);\n-\t}\n-\n-\tfolio_batch_release(\u0026fbatch);\n+\tbvecq_put(roll-\u003etail);\n }\ndiff --git a/fs/netfs/stats.c b/fs/netfs/stats.c\nindex 9a607c4e62dda..0ba6ce9295b25 100644\n--- a/fs/netfs/stats.c\n+++ b/fs/netfs/stats.c\n@@ -46,7 +46,7 @@ atomic_t netfs_n_wh_retry_write_req;\n atomic_t netfs_n_wh_retry_write_subreq;\n atomic_t netfs_n_wb_lock_skip;\n atomic_t netfs_n_wb_lock_wait;\n-atomic_t netfs_n_folioq;\n+atomic_t netfs_n_bvecq;\n \n int netfs_stats_show(struct seq_file *m, void *v)\n {\n@@ -88,10 +88,10 @@ int netfs_stats_show(struct seq_file *m, void *v)\n \t\t atomic_read(\u0026netfs_n_rh_retry_read_subreq),\n \t\t atomic_read(\u0026netfs_n_wh_retry_write_req),\n \t\t atomic_read(\u0026netfs_n_wh_retry_write_subreq));\n-\tseq_printf(m, \"Objs : rr=%u sr=%u foq=%u wsc=%u\\n\",\n+\tseq_printf(m, \"Objs : rr=%u sr=%u bq=%u wsc=%u\\n\",\n \t\t atomic_read(\u0026netfs_n_rh_rreq),\n \t\t atomic_read(\u0026netfs_n_rh_sreq),\n-\t\t atomic_read(\u0026netfs_n_folioq),\n+\t\t atomic_read(\u0026netfs_n_bvecq),\n \t\t atomic_read(\u0026netfs_n_wh_wstream_conflict));\n \tseq_printf(m, \"WbLock : skip=%u wait=%u\\n\",\n \t\t atomic_read(\u0026netfs_n_wb_lock_skip),\ndiff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c\nindex 6e8ea534230df..80e6a3194a609 100644\n--- a/fs/netfs/write_collect.c\n+++ b/fs/netfs/write_collect.c\n@@ -114,11 +114,11 @@ int netfs_folio_written_back(struct folio *folio)\n static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,\n \t\t\t\t\t unsigned int *notes)\n {\n-\tstruct folio_queue *folioq = wreq-\u003ebuffer.tail;\n+\tstruct bvecq *bq = wreq-\u003ebuffer.tail;\n \tunsigned int slot = wreq-\u003ebuffer.first_tail_slot;\n \tuoff_t collected_to = wreq-\u003ecollected_to;\n \n-\tif (WARN_ON_ONCE(!folioq)) {\n+\tif (WARN_ON_ONCE(!bq)) {\n \t\tpr_err(\"[!] Writeback unlock found empty rolling buffer!\\n\");\n \t\tnetfs_dump_request(wreq);\n \t\treturn;\n@@ -130,9 +130,9 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,\n \t\treturn;\n \t}\n \n-\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\tfolioq = rolling_buffer_delete_spent(\u0026wreq-\u003ebuffer);\n-\t\tif (!folioq)\n+\tif (!bvecq_acquire_slot(bq, slot)) {\n+\t\tbq = rolling_buffer_delete_spent(\u0026wreq-\u003ebuffer);\n+\t\tif (!bq)\n \t\t\treturn;\n \t\tslot = 0;\n \t}\n@@ -143,7 +143,7 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,\n \t\tuoff_t fpos, fend;\n \t\tsize_t fsize, flen;\n \n-\t\tfolio = folioq_folio(folioq, slot);\n+\t\tfolio = bvec_folio(\u0026bq-\u003ebv[slot]);\n \t\tif (WARN_ONCE(!folio_test_writeback(folio),\n \t\t\t \"R=%08x: folio %lx is not under writeback\\n\",\n \t\t\t wreq-\u003edebug_id, folio-\u003eindex))\n@@ -166,15 +166,15 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,\n \t\twreq-\u003ecleaned_to = fpos + fsize;\n \t\t*notes |= MADE_PROGRESS;\n \n-\t\t/* Clean up the head folioq. If we clear an entire folioq, then\n-\t\t * we can get rid of it provided it's not also the tail folioq\n+\t\t/* Clean up the head bq. If we clear an entire bq, then\n+\t\t * we can get rid of it provided it's not also the tail bq\n \t\t * being filled by the issuer.\n \t\t */\n-\t\tfolioq_clear(folioq, slot);\n+\t\tbq-\u003ebv[slot].bv_page = NULL;\n \t\tslot++;\n-\t\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\t\tfolioq = rolling_buffer_delete_spent(\u0026wreq-\u003ebuffer);\n-\t\t\tif (!folioq)\n+\t\tif (!bvecq_acquire_slot(bq, slot)) {\n+\t\t\tbq = rolling_buffer_delete_spent(\u0026wreq-\u003ebuffer);\n+\t\t\tif (!bq)\n \t\t\t\tgoto done;\n \t\t\tslot = 0;\n \t\t}\n@@ -183,7 +183,7 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,\n \t\t\tbreak;\n \t}\n \n-\twreq-\u003ebuffer.tail = folioq;\n+\twreq-\u003ebuffer.tail = bq;\n done:\n \twreq-\u003ebuffer.first_tail_slot = slot;\n }\ndiff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c\nindex 3989b4ec0c4b4..9de73c0edad2f 100644\n--- a/fs/netfs/write_issue.c\n+++ b/fs/netfs/write_issue.c\n@@ -107,7 +107,9 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,\n \tictx = netfs_inode(wreq-\u003einode);\n \tif (is_cacheable)\n \t\tfscache_begin_write_operation(\u0026wreq-\u003ecache_resources, netfs_i_cookie(ictx));\n-\tif (rolling_buffer_init(\u0026wreq-\u003ebuffer, wreq-\u003edebug_id, ITER_SOURCE, wreq-\u003egfp) \u003c 0)\n+\tif (rolling_buffer_init(\u0026wreq-\u003ebuffer, ITER_SOURCE, wreq-\u003egfp,\n+\t\t\t\t(origin == NETFS_WRITEBACK ||\n+\t\t\t\t origin == NETFS_WRITEBACK_SINGLE)) \u003c 0)\n \t\tgoto nomem;\n \n \twreq-\u003ecleaned_to = wreq-\u003estart;\n@@ -162,12 +164,12 @@ void netfs_prepare_write(struct netfs_io_request *wreq,\n \tstruct netfs_io_subrequest *subreq;\n \tstruct iov_iter *wreq_iter = \u0026wreq-\u003ebuffer.iter;\n \n-\t/* Make sure we don't point the iterator at a used-up folio_queue\n-\t * struct being used as a placeholder to prevent the queue from\n-\t * collapsing. In such a case, extend the queue.\n+\t/* Make sure we don't point the iterator at a used-up bvecq struct\n+\t * being used as a placeholder to prevent the queue from collapsing.\n+\t * In such a case, extend the queue.\n \t */\n-\tif (iov_iter_is_folioq(wreq_iter) \u0026\u0026\n-\t wreq_iter-\u003efolioq_slot \u003e= folioq_nr_slots(wreq_iter-\u003efolioq))\n+\tif (iov_iter_is_bvecq(wreq_iter) \u0026\u0026\n+\t !bvecq_acquire_slot(wreq_iter-\u003ebvecq, wreq_iter-\u003ebvecq_slot))\n \t\trolling_buffer_make_space(\u0026wreq-\u003ebuffer, wreq-\u003egfp);\n \n \tsubreq = netfs_alloc_subrequest(wreq, stream-\u003esource);\n@@ -450,7 +452,7 @@ static int netfs_write_folio(struct netfs_io_request *wreq,\n \t}\n \n \t/* Attach the folio to the rolling buffer. */\n-\trolling_buffer_append(\u0026wreq-\u003ebuffer, folio, 0, wreq-\u003egfp);\n+\trolling_buffer_append(\u0026wreq-\u003ebuffer, folio, wreq-\u003egfp);\n \n \t/* Move the submission point forward to allow for write-streaming data\n \t * not starting at the front of the page. We don't do write-streaming\n@@ -624,129 +626,11 @@ int netfs_writepages(struct address_space *mapping,\n }\n EXPORT_SYMBOL(netfs_writepages);\n \n-/*\n- * Write some of a pending folio data back to the server and/or the cache.\n- */\n-static int netfs_write_folio_single(struct netfs_io_request *wreq,\n-\t\t\t\t struct folio *folio)\n-{\n-\tstruct netfs_io_stream *upload = \u0026wreq-\u003eio_streams[0];\n-\tstruct netfs_io_stream *cache = \u0026wreq-\u003eio_streams[1];\n-\tstruct netfs_io_stream *stream;\n-\tsize_t iter_off = 0;\n-\tsize_t fsize = folio_size(folio), flen;\n-\tuoff_t fpos = folio_pos(folio);\n-\tssize_t ret;\n-\tbool to_eof = false;\n-\tbool no_debug = false;\n-\n-\t_enter(\"\");\n-\n-\tflen = folio_size(folio);\n-\tif (flen \u003e wreq-\u003ei_size - fpos) {\n-\t\tflen = wreq-\u003ei_size - fpos;\n-\t\tfolio_zero_segment(folio, flen, fsize);\n-\t\tto_eof = true;\n-\t} else if (flen == wreq-\u003ei_size - fpos) {\n-\t\tto_eof = true;\n-\t}\n-\n-\t_debug(\"folio %zx/%zx\", flen, fsize);\n-\n-\tif (!upload-\u003eavail \u0026\u0026 !cache-\u003eavail) {\n-\t\ttrace_netfs_folio(folio, netfs_folio_trace_cancel_store);\n-\t\treturn 0;\n-\t}\n-\n-\tif (!upload-\u003econstruct)\n-\t\ttrace_netfs_folio(folio, netfs_folio_trace_store);\n-\telse\n-\t\ttrace_netfs_folio(folio, netfs_folio_trace_store_plus);\n-\n-\t/* Attach the folio to the rolling buffer. */\n-\tfolio_get(folio);\n-\tret = rolling_buffer_append(\u0026wreq-\u003ebuffer, folio, NETFS_ROLLBUF_PUT_MARK, wreq-\u003egfp);\n-\tif (ret \u003c 0) {\n-\t\tfolio_put(folio);\n-\t\treturn ret;\n-\t}\n-\n-\t/* Move the submission point forward to allow for write-streaming data\n-\t * not starting at the front of the page. We don't do write-streaming\n-\t * with the cache as the cache requires DIO alignment.\n-\t *\n-\t * Also skip uploading for data that's been read and just needs copying\n-\t * to the cache.\n-\t */\n-\tfor (int s = 0; s \u003c NR_IO_STREAMS; s++) {\n-\t\tstream = \u0026wreq-\u003eio_streams[s];\n-\t\tstream-\u003esubmit_off = 0;\n-\t\tstream-\u003esubmit_len = flen;\n-\t\tif (!stream-\u003eavail) {\n-\t\t\tstream-\u003esubmit_off = UINT_MAX;\n-\t\t\tstream-\u003esubmit_len = 0;\n-\t\t}\n-\t}\n-\n-\t/* Attach the folio to one or more subrequests. For a big folio, we\n-\t * could end up with thousands of subrequests if the wsize is small -\n-\t * but we might need to wait during the creation of subrequests for\n-\t * network resources (eg. SMB credits).\n-\t */\n-\tfor (;;) {\n-\t\tssize_t part;\n-\t\tsize_t lowest_off = ULONG_MAX;\n-\t\tint choose_s = -1;\n-\n-\t\t/* Always add to the lowest-submitted stream first. */\n-\t\tfor (int s = 0; s \u003c NR_IO_STREAMS; s++) {\n-\t\t\tstream = \u0026wreq-\u003eio_streams[s];\n-\t\t\tif (stream-\u003esubmit_len \u003e 0 \u0026\u0026\n-\t\t\t stream-\u003esubmit_off \u003c lowest_off) {\n-\t\t\t\tlowest_off = stream-\u003esubmit_off;\n-\t\t\t\tchoose_s = s;\n-\t\t\t}\n-\t\t}\n-\n-\t\tif (choose_s \u003c 0)\n-\t\t\tbreak;\n-\t\tstream = \u0026wreq-\u003eio_streams[choose_s];\n-\n-\t\t/* Advance the iterator(s). */\n-\t\tif (stream-\u003esubmit_off \u003e iter_off) {\n-\t\t\trolling_buffer_advance(\u0026wreq-\u003ebuffer, stream-\u003esubmit_off - iter_off);\n-\t\t\titer_off = stream-\u003esubmit_off;\n-\t\t}\n-\n-\t\tatomic64_set(\u0026wreq-\u003eissued_to, fpos + stream-\u003esubmit_off);\n-\t\tstream-\u003esubmit_extendable_to = fsize - stream-\u003esubmit_off;\n-\t\tpart = netfs_advance_write(wreq, stream, fpos + stream-\u003esubmit_off,\n-\t\t\t\t\t stream-\u003esubmit_len, to_eof);\n-\t\tstream-\u003esubmit_off += part;\n-\t\tif (part \u003e stream-\u003esubmit_len)\n-\t\t\tstream-\u003esubmit_len = 0;\n-\t\telse\n-\t\t\tstream-\u003esubmit_len -= part;\n-\t\tif (part \u003e 0)\n-\t\t\tno_debug = true;\n-\t}\n-\n-\twreq-\u003ebuffer.iter.iov_offset = 0;\n-\tif (fsize \u003e iter_off)\n-\t\trolling_buffer_advance(\u0026wreq-\u003ebuffer, fsize - iter_off);\n-\tatomic64_set(\u0026wreq-\u003eissued_to, fpos + fsize);\n-\n-\tif (!no_debug)\n-\t\tkdebug(\"R=%x: No submit\", wreq-\u003edebug_id);\n-\t_leave(\" = 0\");\n-\treturn 0;\n-}\n-\n /**\n * netfs_writeback_single - Write back a monolithic payload\n * @mapping: The mapping to write from\n * @wbc: Hints from the VM\n- * @iter: Data to write, must be ITER_FOLIOQ.\n+ * @iter: Data to write.\n *\n * Write a monolithic, non-pagecache object back to the server and/or\n * the cache.\n@@ -760,13 +644,8 @@ int netfs_writeback_single(struct address_space *mapping,\n {\n \tstruct netfs_io_request *wreq;\n \tstruct netfs_inode *ictx = netfs_inode(mapping-\u003ehost);\n-\tstruct folio_queue *fq;\n-\tsize_t size = iov_iter_count(iter);\n \tint ret;\n \n-\tif (WARN_ON_ONCE(!iov_iter_is_folioq(iter)))\n-\t\treturn -EIO;\n-\n \tif (!netfs_wb_begin(ictx, wbc-\u003esync_mode == WB_SYNC_NONE)) {\n \t\t/* The VFS will have undirtied the inode. */\n \t\tnetfs_single_mark_inode_dirty(\u0026ictx-\u003einode);\n@@ -779,36 +658,42 @@ int netfs_writeback_single(struct address_space *mapping,\n \t\tgoto couldnt_start;\n \t}\n \n+\twreq-\u003ebuffer.iter = *iter;\n+\twreq-\u003elen = iov_iter_count(iter);\n+\twreq-\u003esubmitted = wreq-\u003elen;\n+\n \t__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, \u0026wreq-\u003eflags);\n \ttrace_netfs_write(wreq, netfs_write_trace_writeback_single);\n \tnetfs_stat(\u0026netfs_n_wh_writepages);\n \n-\tif (__test_and_set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, \u0026wreq-\u003eflags))\n+\tif (test_bit(NETFS_RREQ_UPLOAD_TO_SERVER, \u0026wreq-\u003eflags))\n \t\twreq-\u003enetfs_ops-\u003ebegin_writeback(wreq);\n \n-\tfor (fq = (struct folio_queue *)iter-\u003efolioq; fq; fq = fq-\u003enext) {\n-\t\tfor (int slot = 0; slot \u003c folioq_count(fq); slot++) {\n-\t\t\tstruct folio *folio = folioq_folio(fq, slot);\n-\t\t\tsize_t part = umin(folioq_folio_size(fq, slot), size);\n+\tfor (int s = 0; s \u003c NR_IO_STREAMS; s++) {\n+\t\tstruct netfs_io_subrequest *subreq;\n+\t\tstruct netfs_io_stream *stream = \u0026wreq-\u003eio_streams[s];\n \n-\t\t\t_debug(\"wbiter %lx %llx\", folio-\u003eindex, atomic64_read(\u0026wreq-\u003eissued_to));\n+\t\tif (!stream-\u003eavail)\n+\t\t\tcontinue;\n \n-\t\t\tret = netfs_write_folio_single(wreq, folio);\n-\t\t\tif (ret \u003c 0)\n-\t\t\t\tgoto stop;\n-\t\t\tsize -= part;\n-\t\t\tif (size \u003c= 0)\n-\t\t\t\tgoto stop;\n-\t\t}\n+\t\tnetfs_prepare_write(wreq, stream, 0);\n+\n+\t\tsubreq = stream-\u003econstruct;\n+\t\tsubreq-\u003elen = wreq-\u003elen;\n+\t\tstream-\u003esubmit_len = subreq-\u003elen;\n+\t\tstream-\u003esubmit_extendable_to = round_up(wreq-\u003elen, PAGE_SIZE);\n+\n+\t\tnetfs_issue_write(wreq, stream);\n \t}\n \n-stop:\n-\tfor (int s = 0; s \u003c NR_IO_STREAMS; s++)\n-\t\tnetfs_issue_write(wreq, \u0026wreq-\u003eio_streams[s]);\n \tnetfs_all_subreqs_queued(wreq);\n-\n \tnetfs_wake_collector(wreq);\n \n+\t/* TODO: Might want to be async here if WB_SYNC_NONE, but then need to\n+\t * wait before modifying.\n+\t */\n+\tret = netfs_wait_for_write(wreq);\n+\n \tnetfs_put_request(wreq, netfs_rreq_trace_put_return);\n \t_leave(\" = %d\", ret);\n \treturn ret;\ndiff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h\nindex 79e4e84f8985c..c8b1c23ef4942 100644\n--- a/fs/smb/client/cifsglob.h\n+++ b/fs/smb/client/cifsglob.h\n@@ -289,7 +289,7 @@ struct smb_rqst {\n \tstruct kvec\t*rq_iov;\t/* array of kvecs */\n \tunsigned int\trq_nvec;\t/* number of kvecs in array */\n \tstruct iov_iter\trq_iter;\t/* Data iterator */\n-\tstruct folio_queue *rq_buffer;\t/* Buffer for encryption */\n+\tstruct bvecq\t*rq_buffer;\t/* Buffer for encryption */\n };\n \n struct mid_q_entry;\ndiff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c\nindex aa142420dae2e..317d10b229666 100644\n--- a/fs/smb/client/smb2ops.c\n+++ b/fs/smb/client/smb2ops.c\n@@ -13,7 +13,7 @@\n #include \u003clinux/sort.h\u003e\n #include \u003ccrypto/aead.h\u003e\n #include \u003clinux/fiemap.h\u003e\n-#include \u003clinux/folio_queue.h\u003e\n+#include \u003clinux/bvecq.h\u003e\n #include \u003cuapi/linux/magic.h\u003e\n #include \"cifsfs.h\"\n #include \"cifsglob.h\"\n@@ -4830,19 +4830,18 @@ crypt_message(struct TCP_Server_Info *server, int num_rqst,\n }\n \n /*\n- * Copy data from an iterator to the folios in a folio queue buffer.\n+ * Copy data from an iterator to the pages in a bvec queue buffer.\n */\n-static bool cifs_copy_iter_to_folioq(struct iov_iter *iter, size_t size,\n-\t\t\t\t struct folio_queue *buffer)\n+static bool cifs_copy_iter_to_bvecq(struct iov_iter *iter, size_t size,\n+\t\t\t\t struct bvecq *buffer)\n {\n \tfor (; buffer; buffer = buffer-\u003enext) {\n-\t\tfor (int s = 0; s \u003c folioq_count(buffer); s++) {\n-\t\t\tstruct folio *folio = folioq_folio(buffer, s);\n-\t\t\tsize_t part = folioq_folio_size(buffer, s);\n+\t\tfor (int s = 0; s \u003c bvecq_nr_slots_acquire(buffer); s++) {\n+\t\t\tstruct bio_vec *bv = \u0026buffer-\u003ebv[s];\n+\t\t\tsize_t part = umin(bv-\u003ebv_len, size);\n \n-\t\t\tpart = umin(part, size);\n-\n-\t\t\tif (copy_folio_from_iter(folio, 0, part, iter) != part)\n+\t\t\tif (copy_page_from_iter(bv-\u003ebv_page, bv-\u003ebv_offset,\n+\t\t\t\t\t\tpart, iter) != part)\n \t\t\t\treturn false;\n \t\t\tsize -= part;\n \t\t}\n@@ -4854,7 +4853,7 @@ void\n smb3_free_compound_rqst(int num_rqst, struct smb_rqst *rqst)\n {\n \tfor (int i = 0; i \u003c num_rqst; i++)\n-\t\tnetfs_free_folioq_buffer(rqst[i].rq_buffer);\n+\t\tbvecq_put(rqst[i].rq_buffer);\n }\n \n /*\n@@ -4881,7 +4880,7 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,\n \tfor (int i = 1; i \u003c num_rqst; i++) {\n \t\tstruct smb_rqst *old = \u0026old_rq[i - 1];\n \t\tstruct smb_rqst *new = \u0026new_rq[i];\n-\t\tstruct folio_queue *buffer = NULL;\n+\t\tstruct bvecq *buffer = NULL;\n \t\tsize_t size = iov_iter_count(\u0026old-\u003erq_iter);\n \n \t\torig_len += smb_rqst_len(server, old);\n@@ -4889,17 +4888,16 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,\n \t\tnew-\u003erq_nvec = old-\u003erq_nvec;\n \n \t\tif (size \u003e 0) {\n-\t\t\tsize_t cur_size = 0;\n-\t\t\trc = netfs_alloc_folioq_buffer(NULL, \u0026buffer, \u0026cur_size,\n-\t\t\t\t\t\t size, GFP_NOFS);\n-\t\t\tnew-\u003erq_buffer = buffer;\n-\t\t\tif (rc \u003c 0)\n+\t\t\trc = -ENOMEM;\n+\t\t\tbuffer = bvecq_alloc_buffer(size, GFP_NOFS, true);\n+\t\t\tif (!buffer)\n \t\t\t\tgoto err_free;\n \n-\t\t\tiov_iter_folio_queue(\u0026new-\u003erq_iter, ITER_SOURCE,\n-\t\t\t\t\t buffer, 0, 0, size);\n+\t\t\tnew-\u003erq_buffer = buffer;\n+\t\t\tiov_iter_bvec_queue(\u0026new-\u003erq_iter, ITER_SOURCE,\n+\t\t\t\t\t buffer, 0, 0, size);\n \n-\t\t\tif (!cifs_copy_iter_to_folioq(\u0026old-\u003erq_iter, size, buffer)) {\n+\t\t\tif (!cifs_copy_iter_to_bvecq(\u0026old-\u003erq_iter, size, buffer)) {\n \t\t\t\trc = smb_EIO1(smb_eio_trace_tx_copy_iter_to_buf, size);\n \t\t\t\tgoto err_free;\n \t\t\t}\n@@ -4989,22 +4987,20 @@ decrypt_raw_data(struct TCP_Server_Info *server, char *buf,\n }\n \n static int\n-cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,\n-\t\t\t size_t skip, struct iov_iter *iter)\n+cifs_copy_bvecq_to_iter(struct bvecq *bq, size_t data_size,\n+\t\t\tsize_t skip, struct iov_iter *iter)\n {\n-\tfor (; folioq; folioq = folioq-\u003enext) {\n-\t\tfor (int s = 0; s \u003c folioq_count(folioq); s++) {\n-\t\t\tstruct folio *folio;\n-\t\t\tsize_t fsize, n, len;\n+\tfor (; bq; bq = bq-\u003enext) {\n+\t\tfor (int s = 0; s \u003c bvecq_nr_slots_acquire(bq); s++) {\n+\t\t\tstruct bio_vec *bv = \u0026bq-\u003ebv[s];\n+\t\t\tsize_t n, len;\n \n \t\t\tif (data_size == 0)\n \t\t\t\treturn 0;\n \n-\t\t\tfolio = folioq_folio(folioq, s);\n-\t\t\tfsize = folio_size(folio);\n-\t\t\tlen = umin(fsize - skip, data_size);\n+\t\t\tlen = umin(bv-\u003ebv_len - skip, data_size);\n \n-\t\t\tn = copy_folio_to_iter(folio, skip, len, iter);\n+\t\t\tn = copy_page_to_iter(bv-\u003ebv_page, bv-\u003ebv_offset + skip, len, iter);\n \t\t\tif (n != len) {\n \t\t\t\tcifs_dbg(VFS, \"%s: something went wrong\\n\", __func__);\n \t\t\t\treturn smb_EIO2(smb_eio_trace_rx_copy_to_iter,\n@@ -5026,7 +5022,7 @@ cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,\n \n static int\n handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,\n-\t\t char *buf, unsigned int buf_len, struct folio_queue *buffer,\n+\t\t char *buf, unsigned int buf_len, struct bvecq *buffer,\n \t\t unsigned int buffer_len, bool is_offloaded)\n {\n \tunsigned int data_offset;\n@@ -5136,8 +5132,8 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,\n \t\t}\n \n \t\t/* Copy the data to the output I/O iterator. */\n-\t\trdata-\u003eresult = cifs_copy_folioq_to_iter(buffer, data_len,\n-\t\t\t\t\t\t\t cur_off, \u0026rdata-\u003esubreq.io_iter);\n+\t\trdata-\u003eresult = cifs_copy_bvecq_to_iter(buffer, data_len,\n+\t\t\t\t\t\t\tcur_off, \u0026rdata-\u003esubreq.io_iter);\n \t\tif (rdata-\u003eresult != 0) {\n \t\t\tif (is_offloaded)\n \t\t\t\tmid-\u003emid_state = MID_RESPONSE_MALFORMED;\n@@ -5176,7 +5172,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,\n struct smb2_decrypt_work {\n \tstruct work_struct decrypt;\n \tstruct TCP_Server_Info *server;\n-\tstruct folio_queue *buffer;\n+\tstruct bvecq *buffer;\n \tchar *buf;\n \tunsigned int len;\n };\n@@ -5190,7 +5186,7 @@ static void smb2_decrypt_offload(struct work_struct *work)\n \tstruct mid_q_entry *mid;\n \tstruct iov_iter iter;\n \n-\tiov_iter_folio_queue(\u0026iter, ITER_DEST, dw-\u003ebuffer, 0, 0, dw-\u003elen);\n+\tiov_iter_bvec_queue(\u0026iter, ITER_DEST, dw-\u003ebuffer, 0, 0, dw-\u003elen);\n \trc = decrypt_raw_data(dw-\u003eserver, dw-\u003ebuf, dw-\u003eserver-\u003evals-\u003eread_rsp_size,\n \t\t\t \u0026iter, true);\n \tif (rc) {\n@@ -5239,7 +5235,7 @@ static void smb2_decrypt_offload(struct work_struct *work)\n \t}\n \n free_pages:\n-\tnetfs_free_folioq_buffer(dw-\u003ebuffer);\n+\tbvecq_put(dw-\u003ebuffer);\n \tcifs_small_buf_release(dw-\u003ebuf);\n \tkfree(dw);\n }\n@@ -5285,12 +5281,12 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,\n \tdw-\u003elen = len;\n \tlen = round_up(dw-\u003elen, PAGE_SIZE);\n \n-\tsize_t cur_size = 0;\n-\trc = netfs_alloc_folioq_buffer(NULL, \u0026dw-\u003ebuffer, \u0026cur_size, len, GFP_NOFS);\n-\tif (rc \u003c 0)\n+\trc = -ENOMEM;\n+\tdw-\u003ebuffer = bvecq_alloc_buffer(len, GFP_NOFS, false);\n+\tif (!dw-\u003ebuffer)\n \t\tgoto discard_data;\n \n-\tiov_iter_folio_queue(\u0026iter, ITER_DEST, dw-\u003ebuffer, 0, 0, len);\n+\tiov_iter_bvec_queue(\u0026iter, ITER_DEST, dw-\u003ebuffer, 0, 0, len);\n \n \t/* Read the data into the buffer and clear excess bufferage. */\n \trc = cifs_read_iter_from_socket(server, \u0026iter, dw-\u003elen);\n@@ -5348,7 +5344,7 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,\n \t}\n \n free_pages:\n-\tnetfs_free_folioq_buffer(dw-\u003ebuffer);\n+\tbvecq_put(dw-\u003ebuffer);\n free_dw:\n \tkfree(dw);\n \treturn rc;\ndiff --git a/fs/smb/smbdirect/connection.c b/fs/smb/smbdirect/connection.c\nindex afd31fa12a368..aa52ecdbef2ba 100644\n--- a/fs/smb/smbdirect/connection.c\n+++ b/fs/smb/smbdirect/connection.c\n@@ -5,7 +5,7 @@\n */\n \n #include \"internal.h\"\n-#include \u003clinux/folio_queue.h\u003e\n+#include \u003clinux/bvecq.h\u003e\n \n struct smbdirect_map_sges {\n \tstruct ib_sge *sge;\n@@ -2015,6 +2015,69 @@ static ssize_t smbdirect_map_sges_from_bvec(struct iov_iter *iter,\n \treturn ret;\n }\n \n+/*\n+ * Extract memory fragments from a BVECQ-class iterator and add them to an RDMA\n+ * list. The fragments are not pinned.\n+ */\n+static ssize_t smbdirect_map_sges_from_bvecq(struct iov_iter *iter,\n+\t\t\t\t\t struct smbdirect_map_sges *state,\n+\t\t\t\t\t ssize_t maxsize)\n+{\n+\tconst struct bvecq *bq = iter-\u003ebvecq, *next;\n+\tunsigned int slot = iter-\u003ebvecq_slot;\n+\tssize_t extracted = 0;\n+\tsize_t offset = iter-\u003eiov_offset;\n+\n+\tmaxsize = umin(maxsize, iov_iter_count(iter));\n+\n+\tdo {\n+\t\tstruct bio_vec *bv;\n+\t\tsize_t bsize;\n+\n+\t\twhile (slot \u003e= bq-\u003enr_slots) {\n+\t\t\tnext = bvecq_next(bq);\n+\t\t\tif (!next) {\n+\t\t\t\tif (WARN_ON_ONCE(maxsize \u003e 0))\n+\t\t\t\t\treturn -EIO;\n+\t\t\t\tgoto out;\n+\t\t\t}\n+\t\t\tbq = next;\n+\t\t\tslot = 0;\n+\t\t}\n+\n+\t\tbv = \u0026bq-\u003ebv[slot];\n+\t\tbsize = bv-\u003ebv_len;\n+\n+\t\tif (offset \u003c bsize) {\n+\t\t\tsize_t part = umin(maxsize, bsize - offset);\n+\t\t\tbool ok;\n+\n+\t\t\tok = smbdirect_map_sges_single_page(state,\n+\t\t\t\t\t\t\t bv-\u003ebv_page,\n+\t\t\t\t\t\t\t bv-\u003ebv_offset + offset,\n+\t\t\t\t\t\t\t part);\n+\t\t\tif (!ok)\n+\t\t\t\treturn -EIO;\n+\n+\t\t\toffset += part;\n+\t\t\textracted += part;\n+\t\t\tmaxsize -= part;\n+\t\t}\n+\n+\t\tif (offset \u003e= bsize) {\n+\t\t\toffset = 0;\n+\t\t\tslot++;\n+\t\t}\n+\t} while (state-\u003enum_sge \u003c state-\u003emax_sge \u0026\u0026 maxsize \u003e 0);\n+\n+out:\n+\titer-\u003ebvecq = bq;\n+\titer-\u003ebvecq_slot = slot;\n+\titer-\u003eiov_offset = offset;\n+\titer-\u003ecount -= extracted;\n+\treturn extracted;\n+}\n+\n /*\n * Extract fragments from a KVEC-class iterator and add them to an ib_sge list.\n * This can deal with vmalloc'd buffers as well as kmalloc'd or static buffers.\n@@ -2076,70 +2139,6 @@ static ssize_t smbdirect_map_sges_from_kvec(struct iov_iter *iter,\n \treturn ret;\n }\n \n-/*\n- * Extract folio fragments from a FOLIOQ-class iterator and add them to an\n- * ib_sge list. The folios are not pinned.\n- */\n-static ssize_t smbdirect_map_sges_from_folioq(struct iov_iter *iter,\n-\t\t\t\t\t struct smbdirect_map_sges *state,\n-\t\t\t\t\t ssize_t maxsize)\n-{\n-\tconst struct folio_queue *folioq = iter-\u003efolioq;\n-\tunsigned int slot = iter-\u003efolioq_slot;\n-\tssize_t ret = 0;\n-\tsize_t offset = iter-\u003eiov_offset;\n-\n-\tif (WARN_ON_ONCE(!folioq))\n-\t\treturn -EIO;\n-\n-\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\tfolioq = folioq-\u003enext;\n-\t\tif (WARN_ON_ONCE(!folioq))\n-\t\t\treturn -EIO;\n-\t\tslot = 0;\n-\t}\n-\n-\tdo {\n-\t\tstruct folio *folio = folioq_folio(folioq, slot);\n-\t\tsize_t fsize = folioq_folio_size(folioq, slot);\n-\n-\t\tif (offset \u003c fsize) {\n-\t\t\tsize_t part = umin(maxsize, fsize - offset);\n-\t\t\tbool ok;\n-\n-\t\t\tok = smbdirect_map_sges_single_page(state,\n-\t\t\t\t\t\t\t folio_page(folio, 0),\n-\t\t\t\t\t\t\t offset,\n-\t\t\t\t\t\t\t part);\n-\t\t\tif (!ok)\n-\t\t\t\treturn -EIO;\n-\n-\t\t\toffset += part;\n-\t\t\tret += part;\n-\t\t\tmaxsize -= part;\n-\t\t}\n-\n-\t\tif (offset \u003e= fsize) {\n-\t\t\toffset = 0;\n-\t\t\tslot++;\n-\t\t\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\t\t\tif (!folioq-\u003enext) {\n-\t\t\t\t\tWARN_ON_ONCE(ret \u003c iter-\u003ecount);\n-\t\t\t\t\tbreak;\n-\t\t\t\t}\n-\t\t\t\tfolioq = folioq-\u003enext;\n-\t\t\t\tslot = 0;\n-\t\t\t}\n-\t\t}\n-\t} while (state-\u003enum_sge \u003c state-\u003emax_sge \u0026\u0026 maxsize \u003e 0);\n-\n-\titer-\u003efolioq = folioq;\n-\titer-\u003efolioq_slot = slot;\n-\titer-\u003eiov_offset = offset;\n-\titer-\u003ecount -= ret;\n-\treturn ret;\n-}\n-\n /*\n * Extract page fragments from up to the given amount of the source iterator\n * and build up an ib_sge list that refers to all of those bits. The ib_sge list\n@@ -2164,12 +2163,12 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,\n \tcase ITER_BVEC:\n \t\tret = smbdirect_map_sges_from_bvec(iter, state, len);\n \t\tbreak;\n+\tcase ITER_BVECQ:\n+\t\tret = smbdirect_map_sges_from_bvecq(iter, state, len);\n+\t\tbreak;\n \tcase ITER_KVEC:\n \t\tret = smbdirect_map_sges_from_kvec(iter, state, len);\n \t\tbreak;\n-\tcase ITER_FOLIOQ:\n-\t\tret = smbdirect_map_sges_from_folioq(iter, state, len);\n-\t\tbreak;\n \tdefault:\n \t\tWARN_ONCE(1, \"iov_iter_type[%u]\\n\", iov_iter_type(iter));\n \t\treturn -EIO;\ndiff --git a/include/linux/bvec.h b/include/linux/bvec.h\nindex fc566ee1c1ff1..df9ce860e7874 100644\n--- a/include/linux/bvec.h\n+++ b/include/linux/bvec.h\n@@ -343,4 +343,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)\n \treturn page_to_phys(bvec-\u003ebv_page) + bvec-\u003ebv_offset;\n }\n \n+/**\n+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space\n+ * @bvec: bvec to map\n+ * @offset: Offset into bvec\n+ *\n+ * Map the page containing the byte at @offset into the kernel virtual address\n+ * space. The caller is responsible for making sure this doesn't overrun.\n+ *\n+ * Call kunmap_local on the returned address to unmap.\n+ */\n+static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)\n+{\n+\toffset += bvec-\u003ebv_offset;\n+\n+\treturn kmap_local_page(bvec-\u003ebv_page + (offset \u003e\u003e PAGE_SHIFT)) +\n+\t\t(offset \u0026 ~PAGE_MASK);\n+}\n+\n #endif /* __LINUX_BVEC_H */\ndiff --git a/include/linux/bvecq.h b/include/linux/bvecq.h\nnew file mode 100644\nindex 0000000000000..b984aaa449088\n--- /dev/null\n+++ b/include/linux/bvecq.h\n@@ -0,0 +1,166 @@\n+/* SPDX-License-Identifier: GPL-2.0 */\n+/* Implementation of a segmented queue of bio_vec[].\n+ *\n+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.\n+ * Written by David Howells (dhowells@redhat.com)\n+ */\n+\n+#ifndef _LINUX_BVECQ_H\n+#define _LINUX_BVECQ_H\n+\n+#include \u003clinux/bvec.h\u003e\n+\n+/*\n+ * The type of memory retention used by the elements in bvecq-\u003ebv[] and how to\n+ * clean it up.\n+ */\n+enum bvecq_mem {\n+\tBVECQ_MEM_EXTERNAL,\t/* Externally retained memory - no freeing */\n+\tBVECQ_MEM_PAGECACHE,\t/* Ref'd pagecache pages - must put */\n+\tBVECQ_MEM_GUP,\t\t/* Pinned memory from get_user_pages() - unpin */\n+\tBVECQ_MEM_ALLOCED,\t/* Memory alloc'd by bvecq - can be freed/pooled */\n+} __mode(byte);\n+\n+/*\n+ * Segmented bio_vec queue.\n+ *\n+ * These can be linked together to form messages of indefinite length and\n+ * iterated over with an ITER_BVECQ iterator. The list is non-circular; next\n+ * and prev are NULL at the ends.\n+ *\n+ * The bv pointer points to the bio_vec array; this may be __bv if allocated\n+ * together. The caller is responsible for determining whether or not this is\n+ * the case as the array pointed to by bv may be follow on directly from the\n+ * bvecq by accident of allocation (ie. -\u003ebv == -\u003e__bv is *not* sufficient to\n+ * determine this).\n+ */\n+struct bvecq {\n+\tstruct bvecq\t*next;\t\t/* Next bvec in the list or NULL */\n+\tstruct bvecq\t*prev;\t\t/* Prev bvec in the list or NULL */\n+\trefcount_t\tref;\n+\tu32\t\tpriv;\t\t/* Private data */\n+\tu16\t\tnr_slots;\t/* Number of elements in bv[] used */\n+\tu16\t\tmax_slots;\t/* Number of elements allocated in bv[] */\n+\tenum bvecq_mem\tmem_type:3;\t/* What sort of memory and how to free it */\n+\tbool\t\tinline_bv:1;\t/* T if __bv[] is being used */\n+\tbool\t\tfrom_pool:1;\t/* T if bvecq from mempool */\n+\tstruct bio_vec\t*bv;\t\t/* Pointer to array of page fragments */\n+\tstruct bio_vec\t__bv[];\t\t/* Default array (if -\u003einline_bv) */\n+};\n+\n+/* Number of slots in a 512-byte mempool-backed bvecq. */\n+#define BVECQ_POOL_SLOTS ((512 - sizeof(struct bvecq)) / sizeof(struct bio_vec))\n+\n+/* Number of slots in a 4K bvecq. */\n+#define BVECQ_4KB_SLOTS ((4096 - sizeof(struct bvecq)) / sizeof(struct bio_vec))\n+\n+void bvecq_dump(const struct bvecq *bq);\n+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback);\n+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp, bool for_writeback);\n+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp,\n+\t\t\t\t bool for_writeback);\n+void bvecq_put(struct bvecq *bq);\n+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp);\n+\n+/**\n+ * bvecq_alloc_buffer - Allocate a bvecq chain and populate with buffers\n+ * @size: Target size of the buffer (can be 0 for an empty buffer)\n+ * @gfp: The allocation constraints.\n+ * @for_writeback: True if allocating for writeback\n+ *\n+ * Wrapper around %bvecq_alloc_buffer2().\n+ */\n+static inline struct bvecq *bvecq_alloc_buffer(size_t size, gfp_t gfp, bool for_writeback)\n+{\n+\treturn bvecq_alloc_buffer2(size, 0, gfp, for_writeback);\n+}\n+\n+/**\n+ * bvecq_get - Get a ref on a bvecq\n+ * @bq: The bvecq to get a ref on\n+ */\n+static inline struct bvecq *bvecq_get(struct bvecq *bq)\n+{\n+\trefcount_inc(\u0026bq-\u003eref);\n+\treturn bq;\n+}\n+\n+/**\n+ * bvecq_is_full - Determine if a bvecq is full\n+ * @bvecq: The object to query\n+ *\n+ * Return: true if full; false if not.\n+ */\n+static inline bool bvecq_is_full(const struct bvecq *bvecq)\n+{\n+\treturn bvecq-\u003enr_slots \u003e= bvecq-\u003emax_slots;\n+}\n+\n+/**\n+ * bvecq_filled_to - Release filled slots with release barrier\n+ * @bvecq: The object modified\n+ * @to: The latest slot filled + 1\n+ */\n+static inline void bvecq_filled_to(struct bvecq *bvecq, unsigned int to)\n+{\n+\t/* Set the slot counter after filling the slot */\n+\tsmp_store_release(\u0026bvecq-\u003enr_slots, to);\n+}\n+\n+/**\n+ * bvecq_nr_slots_acquire - Get the number of filled slots with acquire barrier\n+ * @bvecq: The object to query\n+ *\n+ * Return: The number of filled slots\n+ */\n+static inline unsigned int bvecq_nr_slots_acquire(const struct bvecq *bvecq)\n+{\n+\t/* Read the slot counter before looking at the slot */\n+\treturn smp_load_acquire(\u0026bvecq-\u003enr_slots);\n+}\n+\n+/**\n+ * bvecq_acquire_slot - Determine if a slot is valid with acquire barrier\n+ * @bvecq: The object to query\n+ * @slot: The next slot\n+ *\n+ * Return: true if valid; false if might not be valid\n+ */\n+static inline bool bvecq_acquire_slot(const struct bvecq *bvecq, unsigned int slot)\n+{\n+\t/* Read the slot counter before looking at the slot */\n+\treturn slot \u003c bvecq_nr_slots_acquire(bvecq);\n+}\n+\n+/**\n+ * bvecq_append - Get the next bvecq with appropriate barrier\n+ * @to: The bvecq to append to\n+ * @add: The bvecq to append\n+ *\n+ * Attach a new bvecq to a chain using an appropriate barrier to protect the\n+ * write.\n+ *\n+ * [!] Note that this function transfers the caller's ref to the chain.\n+ */\n+static inline void bvecq_append(struct bvecq *to, struct bvecq *add)\n+{\n+\tadd-\u003eprev = to;\n+\n+\t/* Make sure the initialisation is stored before the next pointer. */\n+\tsmp_store_release(\u0026to-\u003enext, add);\n+}\n+\n+/**\n+ * bvecq_next - Get the next bvecq with appropriate barrier\n+ * @bq: The bvecq to start from\n+ *\n+ * Return the next bvecq in a chain, using an appropriate barrier to protect\n+ * the access.\n+ */\n+static inline struct bvecq *bvecq_next(const struct bvecq *bq)\n+{\n+\t/* Read the contents of the next node after the pointer to it. */\n+\treturn smp_load_acquire(\u0026bq-\u003enext);\n+}\n+\n+#endif /* _LINUX_BVECQ_H */\ndiff --git a/include/linux/folio_queue.h b/include/linux/folio_queue.h\ndeleted file mode 100644\nindex f6d5f1f127c9f..0000000000000\n--- a/include/linux/folio_queue.h\n+++ /dev/null\n@@ -1,282 +0,0 @@\n-/* SPDX-License-Identifier: GPL-2.0-or-later */\n-/* Queue of folios definitions\n- *\n- * Copyright (C) 2024 Red Hat, Inc. All Rights Reserved.\n- * Written by David Howells (dhowells@redhat.com)\n- *\n- * See:\n- *\n- *\tDocumentation/core-api/folio_queue.rst\n- *\n- * for a description of the API.\n- */\n-\n-#ifndef _LINUX_FOLIO_QUEUE_H\n-#define _LINUX_FOLIO_QUEUE_H\n-\n-#include \u003clinux/folio_batch.h\u003e\n-#include \u003clinux/mm.h\u003e\n-\n-/*\n- * Segment in a queue of running buffers. Each segment can hold a number of\n- * folios and a portion of the queue can be referenced with the ITER_FOLIOQ\n- * iterator. The possibility exists of inserting non-folio elements into the\n- * queue (such as gaps).\n- *\n- * Explicit prev and next pointers are used instead of a list_head to make it\n- * easier to add segments to tail and remove them from the head without the\n- * need for a lock.\n- */\n-struct folio_queue {\n-\tstruct folio_batch\tvec;\t\t/* Folios in the queue segment */\n-\tu8\t\t\torders[FOLIO_BATCH_SIZE]; /* Order of each folio */\n-\tstruct folio_queue\t*next;\t\t/* Next queue segment or NULL */\n-\tstruct folio_queue\t*prev;\t\t/* Previous queue segment of NULL */\n-\tunsigned long\t\tmarks;\t\t/* 1-bit mark per folio */\n-\tunsigned long\t\tmarks2;\t\t/* Second 1-bit mark per folio */\n-#if FOLIO_BATCH_SIZE \u003e BITS_PER_LONG\n-#error marks is not big enough\n-#endif\n-\tunsigned int\t\trreq_id;\n-\tunsigned int\t\tdebug_id;\n-};\n-\n-/**\n- * folioq_init - Initialise a folio queue segment\n- * @folioq: The segment to initialise\n- * @rreq_id: The request identifier to use in tracelines.\n- *\n- * Initialise a folio queue segment and set an identifier to be used in traces.\n- *\n- * Note that the folio pointers are left uninitialised.\n- */\n-static inline void folioq_init(struct folio_queue *folioq, unsigned int rreq_id)\n-{\n-\tfolio_batch_init(\u0026folioq-\u003evec);\n-\tfolioq-\u003enext = NULL;\n-\tfolioq-\u003eprev = NULL;\n-\tfolioq-\u003emarks = 0;\n-\tfolioq-\u003emarks2 = 0;\n-\tfolioq-\u003erreq_id = rreq_id;\n-\tfolioq-\u003edebug_id = 0;\n-}\n-\n-/**\n- * folioq_nr_slots: Query the capacity of a folio queue segment\n- * @folioq: The segment to query\n- *\n- * Query the number of folios that a particular folio queue segment might hold.\n- * [!] NOTE: This must not be assumed to be the same for every segment!\n- */\n-static inline unsigned int folioq_nr_slots(const struct folio_queue *folioq)\n-{\n-\treturn FOLIO_BATCH_SIZE;\n-}\n-\n-/**\n- * folioq_count: Query the occupancy of a folio queue segment\n- * @folioq: The segment to query\n- *\n- * Query the number of folios that have been added to a folio queue segment.\n- * Note that this is not decreased as folios are removed from a segment.\n- */\n-static inline unsigned int folioq_count(struct folio_queue *folioq)\n-{\n-\treturn folio_batch_count(\u0026folioq-\u003evec);\n-}\n-\n-/**\n- * folioq_full: Query if a folio queue segment is full\n- * @folioq: The segment to query\n- *\n- * Query if a folio queue segment is fully occupied. Note that this does not\n- * change if folios are removed from a segment.\n- */\n-static inline bool folioq_full(struct folio_queue *folioq)\n-{\n-\t//return !folio_batch_space(\u0026folioq-\u003evec);\n-\treturn folioq_count(folioq) \u003e= folioq_nr_slots(folioq);\n-}\n-\n-/**\n- * folioq_is_marked: Check first folio mark in a folio queue segment\n- * @folioq: The segment to query\n- * @slot: The slot number of the folio to query\n- *\n- * Determine if the first mark is set for the folio in the specified slot in a\n- * folio queue segment.\n- */\n-static inline bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot)\n-{\n-\treturn test_bit(slot, \u0026folioq-\u003emarks);\n-}\n-\n-/**\n- * folioq_mark: Set the first mark on a folio in a folio queue segment\n- * @folioq: The segment to modify\n- * @slot: The slot number of the folio to modify\n- *\n- * Set the first mark for the folio in the specified slot in a folio queue\n- * segment.\n- */\n-static inline void folioq_mark(struct folio_queue *folioq, unsigned int slot)\n-{\n-\tset_bit(slot, \u0026folioq-\u003emarks);\n-}\n-\n-/**\n- * folioq_unmark: Clear the first mark on a folio in a folio queue segment\n- * @folioq: The segment to modify\n- * @slot: The slot number of the folio to modify\n- *\n- * Clear the first mark for the folio in the specified slot in a folio queue\n- * segment.\n- */\n-static inline void folioq_unmark(struct folio_queue *folioq, unsigned int slot)\n-{\n-\tclear_bit(slot, \u0026folioq-\u003emarks);\n-}\n-\n-/**\n- * folioq_is_marked2: Check second folio mark in a folio queue segment\n- * @folioq: The segment to query\n- * @slot: The slot number of the folio to query\n- *\n- * Determine if the second mark is set for the folio in the specified slot in a\n- * folio queue segment.\n- */\n-static inline bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot)\n-{\n-\treturn test_bit(slot, \u0026folioq-\u003emarks2);\n-}\n-\n-/**\n- * folioq_mark2: Set the second mark on a folio in a folio queue segment\n- * @folioq: The segment to modify\n- * @slot: The slot number of the folio to modify\n- *\n- * Set the second mark for the folio in the specified slot in a folio queue\n- * segment.\n- */\n-static inline void folioq_mark2(struct folio_queue *folioq, unsigned int slot)\n-{\n-\tset_bit(slot, \u0026folioq-\u003emarks2);\n-}\n-\n-/**\n- * folioq_unmark2: Clear the second mark on a folio in a folio queue segment\n- * @folioq: The segment to modify\n- * @slot: The slot number of the folio to modify\n- *\n- * Clear the second mark for the folio in the specified slot in a folio queue\n- * segment.\n- */\n-static inline void folioq_unmark2(struct folio_queue *folioq, unsigned int slot)\n-{\n-\tclear_bit(slot, \u0026folioq-\u003emarks2);\n-}\n-\n-/**\n- * folioq_append: Add a folio to a folio queue segment\n- * @folioq: The segment to add to\n- * @folio: The folio to add\n- *\n- * Add a folio to the tail of the sequence in a folio queue segment, increasing\n- * the occupancy count and returning the slot number for the folio just added.\n- * The folio size is extracted and stored in the queue and the marks are left\n- * unmodified.\n- *\n- * Note that it's left up to the caller to check that the segment capacity will\n- * not be exceeded and to extend the queue.\n- */\n-static inline unsigned int folioq_append(struct folio_queue *folioq, struct folio *folio)\n-{\n-\tunsigned int slot = folioq-\u003evec.nr++;\n-\n-\tfolioq-\u003evec.folios[slot] = folio;\n-\tfolioq-\u003eorders[slot] = folio_order(folio);\n-\treturn slot;\n-}\n-\n-/**\n- * folioq_append_mark: Add a folio to a folio queue segment\n- * @folioq: The segment to add to\n- * @folio: The folio to add\n- *\n- * Add a folio to the tail of the sequence in a folio queue segment, increasing\n- * the occupancy count and returning the slot number for the folio just added.\n- * The folio size is extracted and stored in the queue, the first mark is set\n- * and and the second and third marks are left unmodified.\n- *\n- * Note that it's left up to the caller to check that the segment capacity will\n- * not be exceeded and to extend the queue.\n- */\n-static inline unsigned int folioq_append_mark(struct folio_queue *folioq, struct folio *folio)\n-{\n-\tunsigned int slot = folioq-\u003evec.nr++;\n-\n-\tfolioq-\u003evec.folios[slot] = folio;\n-\tfolioq-\u003eorders[slot] = folio_order(folio);\n-\tfolioq_mark(folioq, slot);\n-\treturn slot;\n-}\n-\n-/**\n- * folioq_folio: Get a folio from a folio queue segment\n- * @folioq: The segment to access\n- * @slot: The folio slot to access\n- *\n- * Retrieve the folio in the specified slot from a folio queue segment. Note\n- * that no bounds check is made and if the slot hasn't been added into yet, the\n- * pointer will be undefined. If the slot has been cleared, NULL will be\n- * returned.\n- */\n-static inline struct folio *folioq_folio(const struct folio_queue *folioq, unsigned int slot)\n-{\n-\treturn folioq-\u003evec.folios[slot];\n-}\n-\n-/**\n- * folioq_folio_order: Get the order of a folio from a folio queue segment\n- * @folioq: The segment to access\n- * @slot: The folio slot to access\n- *\n- * Retrieve the order of the folio in the specified slot from a folio queue\n- * segment. Note that no bounds check is made and if the slot hasn't been\n- * added into yet, the order returned will be 0.\n- */\n-static inline unsigned int folioq_folio_order(const struct folio_queue *folioq, unsigned int slot)\n-{\n-\treturn folioq-\u003eorders[slot];\n-}\n-\n-/**\n- * folioq_folio_size: Get the size of a folio from a folio queue segment\n- * @folioq: The segment to access\n- * @slot: The folio slot to access\n- *\n- * Retrieve the size of the folio in the specified slot from a folio queue\n- * segment. Note that no bounds check is made and if the slot hasn't been\n- * added into yet, the size returned will be PAGE_SIZE.\n- */\n-static inline size_t folioq_folio_size(const struct folio_queue *folioq, unsigned int slot)\n-{\n-\treturn PAGE_SIZE \u003c\u003c folioq_folio_order(folioq, slot);\n-}\n-\n-/**\n- * folioq_clear: Clear a folio from a folio queue segment\n- * @folioq: The segment to clear\n- * @slot: The folio slot to clear\n- *\n- * Clear a folio from a sequence in a folio queue segment and clear its marks.\n- * The occupancy count is left unchanged.\n- */\n-static inline void folioq_clear(struct folio_queue *folioq, unsigned int slot)\n-{\n-\tfolioq-\u003evec.folios[slot] = NULL;\n-\tfolioq_unmark(folioq, slot);\n-\tfolioq_unmark2(folioq, slot);\n-}\n-\n-#endif /* _LINUX_FOLIO_QUEUE_H */\ndiff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h\nindex f9a17fbbd3980..39ffd5878f505 100644\n--- a/include/linux/iov_iter.h\n+++ b/include/linux/iov_iter.h\n@@ -10,7 +10,7 @@\n \n #include \u003clinux/uio.h\u003e\n #include \u003clinux/bvec.h\u003e\n-#include \u003clinux/folio_queue.h\u003e\n+#include \u003clinux/bvecq.h\u003e\n \n typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,\n \t\t\t void *priv, void *priv2);\n@@ -142,56 +142,65 @@ size_t iterate_bvec(struct iov_iter *iter, size_t len, void *priv, void *priv2,\n }\n \n /*\n- * Handle ITER_FOLIOQ.\n+ * Handle ITER_BVECQ.\n */\n static __always_inline\n-size_t iterate_folioq(struct iov_iter *iter, size_t len, void *priv, void *priv2,\n-\t\t iov_step_f step)\n+size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,\n+\t\t iov_step_f step)\n {\n-\tconst struct folio_queue *folioq = iter-\u003efolioq;\n-\tunsigned int slot = iter-\u003efolioq_slot;\n+\tconst struct bvecq *bq = iter-\u003ebvecq;\n+\tunsigned int slot = iter-\u003ebvecq_slot;\n \tsize_t progress = 0, skip = iter-\u003eiov_offset;\n \n-\tif (slot == folioq_nr_slots(folioq)) {\n-\t\t/* The iterator may have been extended. */\n-\t\tfolioq = folioq-\u003enext;\n-\t\tslot = 0;\n-\t}\n-\n \tdo {\n-\t\tstruct folio *folio = folioq_folio(folioq, slot);\n-\t\tsize_t part, remain = 0, consumed;\n-\t\tsize_t fsize;\n+\t\tconst struct bio_vec *bvec;\n+\t\tstruct page *page;\n+\t\tsize_t poff, plen;\n \t\tvoid *base;\n \n-\t\tif (!folio)\n-\t\t\tbreak;\n+\t\tif (slot \u003e= bq-\u003enr_slots) {\n+\t\t\tif (!bq-\u003enext)\n+\t\t\t\tbreak;\n+\t\t\tbq = bq-\u003enext;\n+\t\t\tslot = 0;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tbvec = \u0026bq-\u003ebv[slot];\n+\t\t/*\n+\t\t * The caller must ensure that a slot with bv_len\u003e0 has a valid\n+\t\t * bv_page.\n+\t\t */\n+\t\tpage = bvec-\u003ebv_page + (bvec-\u003ebv_offset + skip) / PAGE_SIZE;\n+\t\tpoff = (bvec-\u003ebv_offset + skip) % PAGE_SIZE;\n+\t\tplen = min(bvec-\u003ebv_len - skip, len);\n+\n+\t\twhile (plen \u003e 0) {\n+\t\t\tsize_t part, remain, consumed;\n \n-\t\tfsize = folioq_folio_size(folioq, slot);\n-\t\tif (skip \u003c fsize) {\n-\t\t\tbase = kmap_local_folio(folio, skip);\n-\t\t\tpart = umin(len, PAGE_SIZE - skip % PAGE_SIZE);\n+\t\t\tpart = min(plen, PAGE_SIZE - poff);\n+\t\t\tbase = kmap_local_page(page) + poff;\n \t\t\tremain = step(base, progress, part, priv, priv2);\n \t\t\tkunmap_local(base);\n+\n \t\t\tconsumed = part - remain;\n-\t\t\tlen -= consumed;\n \t\t\tprogress += consumed;\n \t\t\tskip += consumed;\n+\t\t\tlen -= consumed;\n+\t\t\tif (!len || remain)\n+\t\t\t\tgoto stop;\n+\t\t\tpage++;\n+\t\t\tpoff = 0;\n+\t\t\tplen -= consumed;\n \t\t}\n-\t\tif (skip \u003e= fsize) {\n-\t\t\tskip = 0;\n-\t\t\tslot++;\n-\t\t\tif (slot == folioq_nr_slots(folioq) \u0026\u0026 folioq-\u003enext) {\n-\t\t\t\tfolioq = folioq-\u003enext;\n-\t\t\t\tslot = 0;\n-\t\t\t}\n-\t\t}\n-\t\tif (remain)\n-\t\t\tbreak;\n+\n+\t\tskip = 0;\n+\t\tslot++;\n \t} while (len);\n \n-\titer-\u003efolioq_slot = slot;\n-\titer-\u003efolioq = folioq;\n+stop:\n+\titer-\u003ebvecq_slot = slot;\n+\titer-\u003ebvecq = bq;\n \titer-\u003eiov_offset = skip;\n \titer-\u003ecount -= progress;\n \treturn progress;\n@@ -306,8 +315,8 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,\n \t\treturn iterate_bvec(iter, len, priv, priv2, step);\n \tif (iov_iter_is_kvec(iter))\n \t\treturn iterate_kvec(iter, len, priv, priv2, step);\n-\tif (iov_iter_is_folioq(iter))\n-\t\treturn iterate_folioq(iter, len, priv, priv2, step);\n+\tif (iov_iter_is_bvecq(iter))\n+\t\treturn iterate_bvecq(iter, len, priv, priv2, step);\n \tif (iov_iter_is_xarray(iter))\n \t\treturn iterate_xarray(iter, len, priv, priv2, step);\n \treturn iterate_discard(iter, len, priv, priv2, step);\n@@ -342,7 +351,7 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,\n * buffer is presented in segments, which for kernel iteration are broken up by\n * physical pages and mapped, with the mapped address being presented.\n *\n- * [!] Note This will only handle BVEC, KVEC, FOLIOQ, XARRAY and DISCARD-type\n+ * [!] Note This will only handle BVEC, KVEC, BVECQ, XARRAY and DISCARD-type\n * iterators; it will not handle UBUF or IOVEC-type iterators.\n *\n * A step functions, @step, must be provided, one for handling mapped kernel\n@@ -370,8 +379,8 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,\n \t\treturn iterate_bvec(iter, len, priv, priv2, step);\n \tif (iov_iter_is_kvec(iter))\n \t\treturn iterate_kvec(iter, len, priv, priv2, step);\n-\tif (iov_iter_is_folioq(iter))\n-\t\treturn iterate_folioq(iter, len, priv, priv2, step);\n+\tif (iov_iter_is_bvecq(iter))\n+\t\treturn iterate_bvecq(iter, len, priv, priv2, step);\n \tif (iov_iter_is_xarray(iter))\n \t\treturn iterate_xarray(iter, len, priv, priv2, step);\n \treturn iterate_discard(iter, len, priv, priv2, step);\ndiff --git a/include/linux/netfs.h b/include/linux/netfs.h\nindex 67e010b6994b3..31a80f4377d75 100644\n--- a/include/linux/netfs.h\n+++ b/include/linux/netfs.h\n@@ -17,13 +17,13 @@\n #include \u003clinux/workqueue.h\u003e\n #include \u003clinux/fs.h\u003e\n #include \u003clinux/pagemap.h\u003e\n+#include \u003clinux/bvecq.h\u003e\n #include \u003clinux/uio.h\u003e\n #include \u003clinux/rolling_buffer.h\u003e\n \n enum netfs_sreq_ref_trace;\n typedef struct mempool mempool_t;\n struct fscache_occupancy;\n-struct folio_queue;\n \n /**\n * folio_start_private_2 - Start an fscache write on a folio. [DEPRECATED]\n@@ -469,18 +469,6 @@ void netfs_end_io_write(struct inode *inode);\n int netfs_start_io_direct(struct inode *inode);\n void netfs_end_io_direct(struct inode *inode);\n \n-/* Miscellaneous APIs. */\n-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,\n-\t\t\t\t unsigned int trace /*enum netfs_folioq_trace*/);\n-void netfs_folioq_free(struct folio_queue *folioq,\n-\t\t unsigned int trace /*enum netfs_trace_folioq*/);\n-\n-/* Buffer wrangling helpers API. */\n-int netfs_alloc_folioq_buffer(struct address_space *mapping,\n-\t\t\t struct folio_queue **_buffer,\n-\t\t\t size_t *_cur_size, ssize_t size, gfp_t gfp);\n-void netfs_free_folioq_buffer(struct folio_queue *fq);\n-\n /* Writeback exclusion API. */\n bool netfs_wb_begin(struct netfs_inode *ictx, bool nowait);\n void netfs_wb_end(struct netfs_inode *ictx);\ndiff --git a/include/linux/rolling_buffer.h b/include/linux/rolling_buffer.h\nindex a97f7cfaacaad..5c0bc4221f010 100644\n--- a/include/linux/rolling_buffer.h\n+++ b/include/linux/rolling_buffer.h\n@@ -8,49 +8,35 @@\n #ifndef _ROLLING_BUFFER_H\n #define _ROLLING_BUFFER_H\n \n-#include \u003clinux/folio_queue.h\u003e\n+#include \u003clinux/bvecq.h\u003e\n #include \u003clinux/uio.h\u003e\n \n /*\n- * Rolling buffer. Whilst the buffer is live and in use, folios and folio\n- * queue segments can be added to one end by one thread and removed from the\n- * other end by another thread. The buffer isn't allowed to be empty; it must\n- * always have at least one folio_queue in it so that neither side has to\n- * modify both queue pointers.\n+ * Rolling buffer. Whilst the buffer is live and in use, folios and bvecq\n+ * segments can be added to one end by one thread and removed from the other\n+ * end by another thread. The buffer isn't allowed to be empty; it must always\n+ * have at least one bvecq in it so that neither side has to modify both queue\n+ * pointers.\n *\n * The iterator in the buffer is extended as buffers are inserted. It can be\n * snapshotted to use a segment of the buffer.\n */\n struct rolling_buffer {\n-\tstruct folio_queue\t*head;\t\t/* Producer's insertion point */\n-\tstruct folio_queue\t*tail;\t\t/* Consumer's removal point */\n+\tstruct bvecq\t\t*head;\t\t/* Producer's insertion point */\n+\tstruct bvecq\t\t*tail;\t\t/* Consumer's removal point */\n \tstruct iov_iter\t\titer;\t\t/* Iterator tracking what's left in the buffer */\n-\tu8\t\t\tnext_head_slot;\t/* Next slot in -\u003ehead */\n \tu8\t\t\tfirst_tail_slot; /* First slot in -\u003etail */\n+\tbool\t\t\tfor_writeback;\t/* T if being used for writeback */\n };\n \n-/*\n- * Snapshot of a rolling buffer.\n- */\n-struct rolling_buffer_snapshot {\n-\tstruct folio_queue\t*curr_folioq;\t/* Queue segment in which current folio resides */\n-\tunsigned char\t\tcurr_slot;\t/* Folio currently being read */\n-\tunsigned char\t\tcurr_order;\t/* Order of folio */\n-};\n-\n-/* Marks to store per-folio in the internal folio_queue structs. */\n-#define ROLLBUF_MARK_1\tBIT(0)\n-#define ROLLBUF_MARK_2\tBIT(1)\n-\n-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,\n-\t\t\tunsigned int direction, gfp_t gfp);\n+int rolling_buffer_init(struct rolling_buffer *roll, unsigned int direction,\n+\t\t\tgfp_t gfp, bool for_writeback);\n int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp);\n ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,\n \t\t\t\t\t struct readahead_control *ractl,\n-\t\t\t\t\t unsigned int rreq_id, gfp_t gfp);\n-ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,\n-\t\t\t unsigned int flags, gfp_t gfp);\n-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll);\n+\t\t\t\t\t gfp_t gfp);\n+ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio, gfp_t gfp);\n+struct bvecq *rolling_buffer_delete_spent(struct rolling_buffer *roll);\n void rolling_buffer_clear(struct rolling_buffer *roll);\n \n static inline void rolling_buffer_advance(struct rolling_buffer *roll, size_t amount)\ndiff --git a/include/linux/uio.h b/include/linux/uio.h\nindex fe2e985d74d24..8d82a8258ef11 100644\n--- a/include/linux/uio.h\n+++ b/include/linux/uio.h\n@@ -11,7 +11,6 @@\n #include \u003cuapi/linux/uio.h\u003e\n \n struct page;\n-struct folio_queue;\n \n typedef unsigned int __bitwise iov_iter_extraction_t;\n \n@@ -26,7 +25,7 @@ enum iter_type {\n \tITER_IOVEC,\n \tITER_BVEC,\n \tITER_KVEC,\n-\tITER_FOLIOQ,\n+\tITER_BVECQ,\n \tITER_XARRAY,\n \tITER_DISCARD,\n };\n@@ -68,7 +67,7 @@ struct iov_iter {\n \t\t\t\tconst struct iovec *__iov;\n \t\t\t\tconst struct kvec *kvec;\n \t\t\t\tconst struct bio_vec *bvec;\n-\t\t\t\tconst struct folio_queue *folioq;\n+\t\t\t\tconst struct bvecq *bvecq;\n \t\t\t\tstruct xarray *xarray;\n \t\t\t\tvoid __user *ubuf;\n \t\t\t};\n@@ -77,7 +76,7 @@ struct iov_iter {\n \t};\n \tunion {\n \t\tunsigned long nr_segs;\n-\t\tu8 folioq_slot;\n+\t\tu16 bvecq_slot;\n \t\tloff_t xarray_start;\n \t};\n };\n@@ -145,9 +144,9 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)\n \treturn iov_iter_type(i) == ITER_DISCARD;\n }\n \n-static inline bool iov_iter_is_folioq(const struct iov_iter *i)\n+static inline bool iov_iter_is_bvecq(const struct iov_iter *i)\n {\n-\treturn iov_iter_type(i) == ITER_FOLIOQ;\n+\treturn iov_iter_type(i) == ITER_BVECQ;\n }\n \n static inline bool iov_iter_is_xarray(const struct iov_iter *i)\n@@ -295,9 +294,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec\n void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec,\n \t\t\tunsigned long nr_segs, size_t count);\n void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);\n-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,\n-\t\t\t const struct folio_queue *folioq,\n-\t\t\t unsigned int first_slot, unsigned int offset, size_t count);\n+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,\n+\t\t\t const struct bvecq *bvecq,\n+\t\t\t unsigned int first_slot, unsigned int offset, size_t count);\n void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xarray *xarray,\n \t\t loff_t start, size_t count);\n ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,\ndiff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h\nindex bf1e1f185b05c..1f92a653d3bfa 100644\n--- a/include/trace/events/netfs.h\n+++ b/include/trace/events/netfs.h\n@@ -244,14 +244,6 @@\n \tEM(netfs_trace_donate_to_next,\t\t\"to-next\")\t\\\n \tE_(netfs_trace_donate_to_deferred_next,\t\"defer-next\")\n \n-#define netfs_folioq_traces\t\t\t\t\t\\\n-\tEM(netfs_trace_folioq_alloc_buffer,\t\"alloc-buf\")\t\\\n-\tEM(netfs_trace_folioq_clear,\t\t\"clear\")\t\\\n-\tEM(netfs_trace_folioq_delete,\t\t\"delete\")\t\\\n-\tEM(netfs_trace_folioq_make_space,\t\"make-space\")\t\\\n-\tEM(netfs_trace_folioq_rollbuf_init,\t\"roll-init\")\t\\\n-\tE_(netfs_trace_folioq_read_progress,\t\"r-progress\")\n-\n #ifndef __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY\n #define __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY\n \n@@ -270,7 +262,6 @@ enum netfs_sreq_ref_trace { netfs_sreq_ref_traces } __mode(byte);\n enum netfs_folio_trace { netfs_folio_traces } __mode(byte);\n enum netfs_collect_contig_trace { netfs_collect_contig_traces } __mode(byte);\n enum netfs_donate_trace { netfs_donate_traces } __mode(byte);\n-enum netfs_folioq_trace { netfs_folioq_traces } __mode(byte);\n \n #endif\n \n@@ -294,7 +285,6 @@ netfs_sreq_ref_traces;\n netfs_folio_traces;\n netfs_collect_contig_traces;\n netfs_donate_traces;\n-netfs_folioq_traces;\n \n /*\n * Now redefine the EM() and E_() macros to map the enums to the strings that\n@@ -398,7 +388,7 @@ TRACE_EVENT(netfs_sreq,\n \t\t __entry-\u003elen\t= sreq-\u003elen;\n \t\t __entry-\u003etransferred = sreq-\u003etransferred;\n \t\t __entry-\u003estart\t= sreq-\u003estart;\n-\t\t __entry-\u003eslot\t= sreq-\u003eio_iter.folioq_slot;\n+\t\t __entry-\u003eslot\t= sreq-\u003eio_iter.bvecq_slot;\n \t\t\t ),\n \n \t TP_printk(\"R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx s=%u e=%d\",\n@@ -773,29 +763,6 @@ TRACE_EVENT(netfs_collect_stream,\n \t\t __entry-\u003ecollected_to, __entry-\u003eissued_to)\n \t );\n \n-TRACE_EVENT(netfs_folioq,\n-\t TP_PROTO(const struct folio_queue *fq,\n-\t\t enum netfs_folioq_trace trace),\n-\n-\t TP_ARGS(fq, trace),\n-\n-\t TP_STRUCT__entry(\n-\t\t __field(unsigned int,\t\trreq)\n-\t\t __field(unsigned int,\t\tid)\n-\t\t __field(enum netfs_folioq_trace,\ttrace)\n-\t\t\t ),\n-\n-\t TP_fast_assign(\n-\t\t __entry-\u003erreq\t= fq ? fq-\u003erreq_id : 0;\n-\t\t __entry-\u003eid\t\t= fq ? fq-\u003edebug_id : 0;\n-\t\t __entry-\u003etrace\t= trace;\n-\t\t\t ),\n-\n-\t TP_printk(\"R=%08x fq=%x %s\",\n-\t\t __entry-\u003erreq, __entry-\u003eid,\n-\t\t __print_symbolic(__entry-\u003etrace, netfs_folioq_traces))\n-\t );\n-\n TRACE_EVENT(netfs_read_progress_at,\n \t TP_PROTO(const struct netfs_io_request *rreq),\n \ndiff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c\nindex d870bc5e50bc8..63fd6156ffa7c 100644\n--- a/kernel/bpf/btf.c\n+++ b/kernel/bpf/btf.c\n@@ -6803,8 +6803,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {\n \t/* amdgpu */\n \t{ \"amdgpu_vm_bo_map\", 0x1 },\n \t{ \"amdgpu_vm_bo_unmap\", 0x1 },\n-\t/* netfs */\n-\t{ \"netfs_folioq\", 0x1 },\n \t/* xfs from xfs_defer_pending_class */\n \t{ \"xfs_defer_create_intent\", 0x1 },\n \t{ \"xfs_defer_cancel_list\", 0x1 },\ndiff --git a/lib/iov_iter.c b/lib/iov_iter.c\nindex 2072c04e99d05..7dcb81ba2a73a 100644\n--- a/lib/iov_iter.c\n+++ b/lib/iov_iter.c\n@@ -538,37 +538,38 @@ static void iov_iter_iovec_advance(struct iov_iter *i, size_t size)\n \ti-\u003e__iov = iov;\n }\n \n-static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)\n+static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)\n {\n-\tconst struct folio_queue *folioq = i-\u003efolioq;\n-\tunsigned int slot = i-\u003efolioq_slot;\n+\tconst struct bvecq *bq = i-\u003ebvecq;\n+\tunsigned int slot = i-\u003ebvecq_slot;\n \n \tif (!i-\u003ecount)\n \t\treturn;\n-\ti-\u003ecount -= size;\n-\n-\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\tfolioq = folioq-\u003enext;\n-\t\tslot = 0;\n-\t}\n+\ti-\u003ecount -= by;\n \n-\tsize += i-\u003eiov_offset; /* From beginning of current segment. */\n+\tby += i-\u003eiov_offset; /* From beginning of current segment. */\n \tdo {\n-\t\tsize_t fsize = folioq_folio_size(folioq, slot);\n+\t\tsize_t len;\n \n-\t\tif (likely(size \u003c fsize))\n-\t\t\tbreak;\n-\t\tsize -= fsize;\n-\t\tslot++;\n-\t\tif (slot \u003e= folioq_nr_slots(folioq) \u0026\u0026 folioq-\u003enext) {\n-\t\t\tfolioq = folioq-\u003enext;\n+\t\tif (slot \u003e= bq-\u003enr_slots) {\n+\t\t\tif (!bq-\u003enext)\n+\t\t\t\tbreak;\n+\t\t\tbq = bq-\u003enext;\n \t\t\tslot = 0;\n+\t\t\tcontinue;\n \t\t}\n-\t} while (size);\n \n-\ti-\u003eiov_offset = size;\n-\ti-\u003efolioq_slot = slot;\n-\ti-\u003efolioq = folioq;\n+\t\tlen = bq-\u003ebv[slot].bv_len;\n+\n+\t\tif (likely(by \u003c len))\n+\t\t\tbreak;\n+\t\tby -= len;\n+\t\tslot++;\n+\t} while (by);\n+\n+\ti-\u003eiov_offset = by;\n+\ti-\u003ebvecq_slot = slot;\n+\ti-\u003ebvecq = bq;\n }\n \n void iov_iter_advance(struct iov_iter *i, size_t size)\n@@ -583,38 +584,39 @@ void iov_iter_advance(struct iov_iter *i, size_t size)\n \t\tiov_iter_iovec_advance(i, size);\n \t} else if (iov_iter_is_bvec(i)) {\n \t\tiov_iter_bvec_advance(i, size);\n-\t} else if (iov_iter_is_folioq(i)) {\n-\t\tiov_iter_folioq_advance(i, size);\n+\t} else if (iov_iter_is_bvecq(i)) {\n+\t\tiov_iter_bvecq_advance(i, size);\n \t} else if (iov_iter_is_discard(i)) {\n \t\ti-\u003ecount -= size;\n \t}\n }\n EXPORT_SYMBOL(iov_iter_advance);\n \n-static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)\n+static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)\n {\n-\tconst struct folio_queue *folioq = i-\u003efolioq;\n-\tunsigned int slot = i-\u003efolioq_slot;\n+\tconst struct bvecq *bq = i-\u003ebvecq;\n+\tunsigned int slot = i-\u003ebvecq_slot;\n \n \tfor (;;) {\n-\t\tsize_t fsize;\n+\t\tsize_t len;\n \n \t\tif (slot == 0) {\n-\t\t\tfolioq = folioq-\u003eprev;\n-\t\t\tslot = folioq_nr_slots(folioq);\n+\t\t\tbq = bq-\u003eprev;\n+\t\t\tslot = bq-\u003enr_slots;\n+\t\t\tcontinue;\n \t\t}\n \t\tslot--;\n \n-\t\tfsize = folioq_folio_size(folioq, slot);\n-\t\tif (unroll \u003c= fsize) {\n-\t\t\ti-\u003eiov_offset = fsize - unroll;\n+\t\tlen = bq-\u003ebv[slot].bv_len;\n+\t\tif (unroll \u003c= len) {\n+\t\t\ti-\u003eiov_offset = len - unroll;\n \t\t\tbreak;\n \t\t}\n-\t\tunroll -= fsize;\n+\t\tunroll -= len;\n \t}\n \n-\ti-\u003efolioq_slot = slot;\n-\ti-\u003efolioq = folioq;\n+\ti-\u003ebvecq_slot = slot;\n+\ti-\u003ebvecq = bq;\n }\n \n void iov_iter_revert(struct iov_iter *i, size_t unroll)\n@@ -648,9 +650,9 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)\n \t\t\t}\n \t\t\tunroll -= n;\n \t\t}\n-\t} else if (iov_iter_is_folioq(i)) {\n+\t} else if (iov_iter_is_bvecq(i)) {\n \t\ti-\u003eiov_offset = 0;\n-\t\tiov_iter_folioq_revert(i, unroll);\n+\t\tiov_iter_bvecq_revert(i, unroll);\n \t} else { /* same logics for iovec and kvec */\n \t\tconst struct iovec *iov = iter_iov(i);\n \t\twhile (1) {\n@@ -678,9 +680,28 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)\n \t\tif (iov_iter_is_bvec(i))\n \t\t\treturn min(i-\u003ecount, i-\u003ebvec-\u003ebv_len - i-\u003eiov_offset);\n \t}\n-\tif (unlikely(iov_iter_is_folioq(i)))\n-\t\treturn !i-\u003ecount ? 0 :\n-\t\t\tumin(folioq_folio_size(i-\u003efolioq, i-\u003efolioq_slot), i-\u003ecount);\n+\tif (!i-\u003ecount)\n+\t\treturn 0;\n+\tif (unlikely(iov_iter_is_bvecq(i))) {\n+\t\tconst struct bvecq *bq = i-\u003ebvecq;\n+\t\tunsigned int slot = i-\u003ebvecq_slot;\n+\t\tsize_t offset = i-\u003eiov_offset;\n+\n+\t\tfor (;;) {\n+\t\t\twhile (slot \u003e= bq-\u003enr_slots) {\n+\t\t\t\tbq = bq-\u003enext;\n+\t\t\t\tif (!bq)\n+\t\t\t\t\treturn 0;\n+\t\t\t\tslot = 0;\n+\t\t\t\toffset = 0;\n+\t\t\t}\n+\t\t\tif (bq-\u003ebv[slot].bv_len \u003e offset)\n+\t\t\t\tbreak;\n+\t\t\tslot++;\n+\t\t\toffset = 0;\n+\t\t}\n+\t\treturn min(i-\u003ecount, bq-\u003ebv[slot].bv_len - offset);\n+\t}\n \treturn i-\u003ecount;\n }\n EXPORT_SYMBOL(iov_iter_single_seg_count);\n@@ -718,34 +739,33 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction,\n EXPORT_SYMBOL(iov_iter_bvec);\n \n /**\n- * iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue\n+ * iov_iter_bvec_queue - Initialise an I/O iterator to use a segmented bvec queue\n * @i: The iterator to initialise.\n * @direction: The direction of the transfer.\n- * @folioq: The starting point in the folio queue.\n- * @first_slot: The first slot in the folio queue to use\n- * @offset: The offset into the folio in the first slot to start at\n+ * @bvecq: The starting point in the bvec queue.\n+ * @first_slot: The first slot in the bvec queue to use\n+ * @offset: The offset into the bvec in the first slot to start at\n * @count: The size of the I/O buffer in bytes.\n *\n- * Set up an I/O iterator to either draw data out of the pages attached to an\n- * inode or to inject data into those pages. The pages *must* be prevented\n- * from evaporation, either by taking a ref on them or locking them by the\n- * caller.\n+ * Set up an I/O iterator to either draw data out of the buffers attached to an\n+ * inode or to inject data into those buffers. The pages *must* be prevented\n+ * from evaporation, either by the caller.\n */\n-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,\n-\t\t\t const struct folio_queue *folioq, unsigned int first_slot,\n-\t\t\t unsigned int offset, size_t count)\n+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,\n+\t\t\t const struct bvecq *bvecq, unsigned int first_slot,\n+\t\t\t unsigned int offset, size_t count)\n {\n-\tBUG_ON(direction \u0026 ~1);\n+\tWARN_ON(direction \u0026 ~(READ | WRITE));\n \t*i = (struct iov_iter) {\n-\t\t.iter_type = ITER_FOLIOQ,\n-\t\t.data_source = direction,\n-\t\t.folioq = folioq,\n-\t\t.folioq_slot = first_slot,\n-\t\t.count = count,\n-\t\t.iov_offset = offset,\n+\t\t.iter_type\t= ITER_BVECQ,\n+\t\t.data_source\t= direction,\n+\t\t.bvecq\t\t= bvecq,\n+\t\t.bvecq_slot\t= first_slot,\n+\t\t.count\t\t= count,\n+\t\t.iov_offset\t= offset,\n \t};\n }\n-EXPORT_SYMBOL(iov_iter_folio_queue);\n+EXPORT_SYMBOL(iov_iter_bvec_queue);\n \n /**\n * iov_iter_xarray - Initialise an I/O iterator to use the pages in an xarray\n@@ -839,6 +859,39 @@ static unsigned long iov_iter_alignment_bvec(const struct iov_iter *i)\n \treturn res;\n }\n \n+static unsigned long iov_iter_alignment_bvecq(const struct iov_iter *iter)\n+{\n+\tconst struct bvecq *bq;\n+\tunsigned long res = 0;\n+\tunsigned int slot = iter-\u003ebvecq_slot;\n+\tsize_t skip = iter-\u003eiov_offset;\n+\tsize_t size = iter-\u003ecount;\n+\n+\tif (!size)\n+\t\treturn res;\n+\n+\tfor (bq = iter-\u003ebvecq; bq; bq = bq-\u003enext) {\n+\t\tfor (; slot \u003c bq-\u003enr_slots; slot++) {\n+\t\t\tconst struct bio_vec *bvec = \u0026bq-\u003ebv[slot];\n+\t\t\tsize_t part = min(bvec-\u003ebv_len - skip, size);\n+\n+\t\t\tif (part) {\n+\t\t\t\tres |= bvec-\u003ebv_offset + skip;\n+\t\t\t\tres |= part;\n+\t\t\t}\n+\n+\t\t\tsize -= part;\n+\t\t\tif (size == 0)\n+\t\t\t\treturn res;\n+\t\t\tskip = 0;\n+\t\t}\n+\n+\t\tslot = 0;\n+\t}\n+\n+\treturn res;\n+}\n+\n unsigned long iov_iter_alignment(const struct iov_iter *i)\n {\n \tif (likely(iter_is_ubuf(i))) {\n@@ -854,10 +907,10 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)\n \n \tif (iov_iter_is_bvec(i))\n \t\treturn iov_iter_alignment_bvec(i);\n+\tif (iov_iter_is_bvecq(i))\n+\t\treturn iov_iter_alignment_bvecq(i);\n \n-\t/* With both xarray and folioq types, we're dealing with whole folios. */\n-\tif (iov_iter_is_folioq(i))\n-\t\treturn i-\u003eiov_offset | i-\u003ecount;\n+\t/* With the xarray type, we're dealing with whole folios. */\n \tif (iov_iter_is_xarray(i))\n \t\treturn (i-\u003exarray_start + i-\u003eiov_offset) | i-\u003ecount;\n \n@@ -910,62 +963,156 @@ static int want_pages_array(struct page ***res, size_t size,\n \treturn count;\n }\n \n-static ssize_t iter_folioq_get_pages(struct iov_iter *iter,\n-\t\t\t\t struct page ***ppages, size_t maxsize,\n-\t\t\t\t unsigned maxpages, size_t *_start_offset)\n+/*\n+ * Count the number of virtually contiguous pages coming up next in an\n+ * ITER_BVECQ iterator, up to the specified maxima.\n+ */\n+static unsigned int iter_count_bvecq_pages(const struct iov_iter *iter,\n+\t\t\t\t\t size_t maxsize,\n+\t\t\t\t\t unsigned int maxpages)\n {\n-\tconst struct folio_queue *folioq = iter-\u003efolioq;\n+\tconst struct bvecq *bvecq = iter-\u003ebvecq;\n+\tunsigned int slot = iter-\u003ebvecq_slot;\n+\tssize_t remain = umin(maxsize, iter-\u003ecount);\n+\tsize_t count = 0, offset = iter-\u003eiov_offset;\n+\n+\tdo {\n+\t\tconst struct bio_vec *bv;\n+\t\tsize_t boff, blen;\n+\n+\t\tif (slot \u003e= bvecq-\u003enr_slots) {\n+\t\t\tif (!bvecq-\u003enext) {\n+\t\t\t\tWARN_ON_ONCE(remain \u003e 0);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tbvecq = bvecq-\u003enext;\n+\t\t\tslot = 0;\n+\t\t\toffset = 0;\n+\t\t\tcontinue;\n+\t\t}\n+\n+\t\tbv = \u0026bvecq-\u003ebv[slot++];\n+\t\tboff = bv-\u003ebv_offset;\n+\t\tblen = bv-\u003ebv_len;\n+\n+\t\t/* bv_page is not allowed to be NULL unless bv_len == 0. */\n+\t\tif (WARN_ON_ONCE(!bv-\u003ebv_page \u0026\u0026 blen \u003e 0))\n+\t\t\tbreak;\n+\t\tif (!PAGE_ALIGNED(boff) \u0026\u0026 count \u003e 0)\n+\t\t\tbreak;\n+\n+\t\tboff += offset;\n+\t\tblen -= offset;\n+\t\toffset = 0;\n+\t\tif (!blen)\n+\t\t\tcontinue;\n+\n+\t\tblen = umin(blen, remain);\n+\t\tremain -= blen;\n+\t\tblen += offset_in_page(boff);\n+\t\tcount += DIV_ROUND_UP(blen, PAGE_SIZE);\n+\n+\t\tif (!PAGE_ALIGNED(blen))\n+\t\t\tbreak;\n+\t} while (remain \u003e 0 \u0026\u0026 count \u003c maxpages);\n+\n+\treturn min(count, maxpages);\n+}\n+\n+/*\n+ * Get a list of virtually contiguous pages from an ITER_BVECQ iterator and pin\n+ * them.\n+ */\n+static ssize_t iter_bvecq_get_pages(struct iov_iter *iter,\n+\t\t\t\t struct page ***ppages, size_t maxsize,\n+\t\t\t\t unsigned int maxpages, size_t *_start_offset)\n+{\n+\tconst struct bvecq *bvecq = iter-\u003ebvecq;\n \tstruct page **pages;\n-\tunsigned int slot = iter-\u003efolioq_slot;\n-\tsize_t extracted = 0, count = iter-\u003ecount, iov_offset = iter-\u003eiov_offset;\n+\tunsigned int slot = iter-\u003ebvecq_slot, nr = 0;\n+\tsize_t extracted = 0, offset = iter-\u003eiov_offset;\n \n-\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\tfolioq = folioq-\u003enext;\n-\t\tslot = 0;\n-\t\tif (WARN_ON(iov_offset != 0))\n-\t\t\treturn -EIO;\n+\tfor (;;) {\n+\t\tif (slot \u003e= bvecq-\u003enr_slots) {\n+\t\t\tif (!bvecq-\u003enext)\n+\t\t\t\treturn 0;\n+\t\t\tbvecq = bvecq-\u003enext;\n+\t\t\tslot = 0;\n+\t\t\toffset = 0;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tif (offset \u003e= bvecq-\u003ebv[slot].bv_len) {\n+\t\t\tslot++;\n+\t\t\toffset = 0;\n+\t\t\tcontinue;\n+\t\t}\n+\t\tbreak;\n \t}\n \n-\tmaxpages = want_pages_array(ppages, maxsize, iov_offset \u0026 ~PAGE_MASK, maxpages);\n+\t/* Count the next run of virtually contiguous pages. */\n+\tmaxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);\n+\tif (!maxpages)\n+\t\treturn 0;\n+\n+\tmaxpages = want_pages_array(ppages, maxsize,\n+\t\t\t\t (bvecq-\u003ebv[slot].bv_offset + offset) \u0026 ~PAGE_MASK,\n+\t\t\t\t maxpages);\n \tif (!maxpages)\n \t\treturn -ENOMEM;\n-\t*_start_offset = iov_offset \u0026 ~PAGE_MASK;\n \tpages = *ppages;\n \n-\tfor (;;) {\n-\t\tstruct folio *folio = folioq_folio(folioq, slot);\n-\t\tsize_t offset = iov_offset, fsize = folioq_folio_size(folioq, slot);\n-\t\tsize_t part = PAGE_SIZE - offset % PAGE_SIZE;\n-\n-\t\tif (offset \u003c fsize) {\n-\t\t\tpart = umin(part, umin(maxsize - extracted, fsize - offset));\n-\t\t\tcount -= part;\n-\t\t\tiov_offset += part;\n-\t\t\textracted += part;\n+\t/* Now transcribe the page pointers. */\n+\tdo {\n+\t\tconst struct bio_vec *bv;\n+\t\tsize_t boff, blen;\n \n-\t\t\t*pages = folio_page(folio, offset / PAGE_SIZE);\n-\t\t\tget_page(*pages);\n-\t\t\tpages++;\n-\t\t\tmaxpages--;\n+\t\tif (slot \u003e= bvecq-\u003enr_slots) {\n+\t\t\tif (!bvecq-\u003enext) {\n+\t\t\t\tWARN_ON_ONCE(extracted \u003c iter-\u003ecount);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tbvecq = bvecq-\u003enext;\n+\t\t\tslot = 0;\n+\t\t\toffset = 0;\n+\t\t\tcontinue;\n \t\t}\n \n-\t\tif (maxpages == 0 || extracted \u003e= maxsize)\n-\t\t\tbreak;\n+\t\tbv = \u0026bvecq-\u003ebv[slot];\n+\t\tboff = bv-\u003ebv_offset;\n+\t\tblen = bv-\u003ebv_len;\n+\n+\t\t/* bv_page is not allowed to be NULL unless bv_len == 0. */\n+\n+\t\tif (offset \u003c blen) {\n+\t\t\tsize_t poff = (boff + offset) % PAGE_SIZE;\n+\t\t\tsize_t part = min(maxsize - extracted, blen - offset);\n+\t\t\tsize_t pix = (boff + offset) / PAGE_SIZE;\n+\n+\t\t\tif (poff + part \u003e PAGE_SIZE)\n+\t\t\t\tpart = PAGE_SIZE - poff;\n+\n+\t\t\tif (!extracted)\n+\t\t\t\t*_start_offset = poff;\n \n-\t\tif (iov_offset \u003e= fsize) {\n-\t\t\tiov_offset = 0;\n+\t\t\t/* Get a ref on the page if it's not a slab page. */\n+\t\t\tif (!PageSlab(bv-\u003ebv_page + pix))\n+\t\t\t\tget_page(bv-\u003ebv_page + pix);\n+\n+\t\t\tpages[nr++] = bv-\u003ebv_page + pix;\n+\t\t\toffset += part;\n+\t\t\textracted += part;\n+\t\t}\n+\n+\t\tif (offset \u003e= blen) {\n+\t\t\toffset = 0;\n \t\t\tslot++;\n-\t\t\tif (slot == folioq_nr_slots(folioq) \u0026\u0026 folioq-\u003enext) {\n-\t\t\t\tfolioq = folioq-\u003enext;\n-\t\t\t\tslot = 0;\n-\t\t\t}\n \t\t}\n-\t}\n+\t} while (nr \u003c maxpages \u0026\u0026 extracted \u003c maxsize);\n \n-\titer-\u003ecount = count;\n-\titer-\u003eiov_offset = iov_offset;\n-\titer-\u003efolioq = folioq;\n-\titer-\u003efolioq_slot = slot;\n+\titer-\u003ebvecq = bvecq;\n+\titer-\u003ebvecq_slot = slot;\n+\titer-\u003eiov_offset = offset;\n+\titer-\u003ecount -= extracted;\n \treturn extracted;\n }\n \n@@ -1120,8 +1267,8 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,\n \t\t}\n \t\treturn maxsize;\n \t}\n-\tif (iov_iter_is_folioq(i))\n-\t\treturn iter_folioq_get_pages(i, pages, maxsize, maxpages, start);\n+\tif (iov_iter_is_bvecq(i))\n+\t\treturn iter_bvecq_get_pages(i, pages, maxsize, maxpages, start);\n \tif (iov_iter_is_xarray(i))\n \t\treturn iter_xarray_get_pages(i, pages, maxsize, maxpages, start);\n \treturn -EFAULT;\n@@ -1192,6 +1339,38 @@ static int bvec_npages(const struct iov_iter *i, int maxpages)\n \treturn npages;\n }\n \n+static size_t iov_npages_bvecq(const struct iov_iter *iter, size_t maxpages)\n+{\n+\tconst struct bvecq *bq;\n+\tunsigned int slot = iter-\u003ebvecq_slot;\n+\tsize_t npages = 0;\n+\tsize_t skip = iter-\u003eiov_offset;\n+\tsize_t size = iter-\u003ecount;\n+\n+\tfor (bq = iter-\u003ebvecq; bq; bq = bq-\u003enext) {\n+\t\tfor (; slot \u003c bq-\u003enr_slots; slot++) {\n+\t\t\tconst struct bio_vec *bvec = \u0026bq-\u003ebv[slot];\n+\t\t\tsize_t offs = (bvec-\u003ebv_offset + skip) % PAGE_SIZE;\n+\t\t\tsize_t part = min(bvec-\u003ebv_len - skip, size);\n+\n+\t\t\tif (part) {\n+\t\t\t\tnpages += DIV_ROUND_UP(offs + part, PAGE_SIZE);\n+\t\t\t\tif (npages \u003e= maxpages)\n+\t\t\t\t\tgoto out;\n+\t\t\t}\n+\n+\t\t\tsize -= part;\n+\t\t\tif (!size)\n+\t\t\t\tgoto out;\n+\t\t\tskip = 0;\n+\t\t}\n+\n+\t\tslot = 0;\n+\t}\n+out:\n+\treturn umin(npages, maxpages);\n+}\n+\n int iov_iter_npages(const struct iov_iter *i, int maxpages)\n {\n \tif (unlikely(!i-\u003ecount))\n@@ -1206,11 +1385,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)\n \t\treturn iov_npages(i, maxpages);\n \tif (iov_iter_is_bvec(i))\n \t\treturn bvec_npages(i, maxpages);\n-\tif (iov_iter_is_folioq(i)) {\n-\t\tunsigned offset = i-\u003eiov_offset % PAGE_SIZE;\n-\t\tint npages = DIV_ROUND_UP(offset + i-\u003ecount, PAGE_SIZE);\n-\t\treturn min(npages, maxpages);\n-\t}\n+\tif (iov_iter_is_bvecq(i))\n+\t\treturn iov_npages_bvecq(i, maxpages);\n \tif (iov_iter_is_xarray(i)) {\n \t\tunsigned offset = (i-\u003exarray_start + i-\u003eiov_offset) % PAGE_SIZE;\n \t\tint npages = DIV_ROUND_UP(offset + i-\u003ecount, PAGE_SIZE);\n@@ -1494,64 +1670,86 @@ void iov_iter_restore(struct iov_iter *i, struct iov_iter_state *state)\n EXPORT_SYMBOL_FOR_MODULES(iov_iter_restore, \"vmw_vsock_virtio_transport_common\");\n \n /*\n- * Extract a list of contiguous pages from an ITER_FOLIOQ iterator. This does\n- * not get references on the pages, nor does it get a pin on them.\n+ * Extract a list of virtually contiguous pages from an ITER_BVECQ iterator.\n+ * This does not get references on the pages, nor does it get a pin on them.\n */\n-static ssize_t iov_iter_extract_folioq_pages(struct iov_iter *i,\n-\t\t\t\t\t struct page ***pages, size_t maxsize,\n-\t\t\t\t\t unsigned int maxpages,\n-\t\t\t\t\t iov_iter_extraction_t extraction_flags,\n-\t\t\t\t\t size_t *offset0)\n+static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,\n+\t\t\t\t\t struct page ***pages, size_t maxsize,\n+\t\t\t\t\t unsigned int maxpages,\n+\t\t\t\t\t iov_iter_extraction_t extraction_flags,\n+\t\t\t\t\t size_t *offset0)\n {\n-\tconst struct folio_queue *folioq = i-\u003efolioq;\n+\tconst struct bvecq *bvecq;\n \tstruct page **p;\n-\tunsigned int nr = 0;\n-\tsize_t extracted = 0, offset, slot = i-\u003efolioq_slot;\n+\tunsigned int slot, nr = 0;\n+\tsize_t extracted = 0, offset;\n \n-\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\tfolioq = folioq-\u003enext;\n-\t\tslot = 0;\n-\t\tif (WARN_ON(i-\u003eiov_offset != 0))\n-\t\t\treturn -EIO;\n-\t}\n+\t/* Count the next run of virtually contiguous pages. */\n+\tmaxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);\n+\tif (!maxpages)\n+\t\treturn 0;\n \n-\toffset = i-\u003eiov_offset \u0026 ~PAGE_MASK;\n-\t*offset0 = offset;\n+\tif (!*pages) {\n+\t\t*pages = kvmalloc_array(maxpages, sizeof(struct page *), GFP_KERNEL);\n+\t\tif (!*pages)\n+\t\t\treturn -ENOMEM;\n+\t}\n \n-\tmaxpages = want_pages_array(pages, maxsize, offset, maxpages);\n-\tif (!maxpages)\n-\t\treturn -ENOMEM;\n \tp = *pages;\n \n-\tfor (;;) {\n-\t\tstruct folio *folio = folioq_folio(folioq, slot);\n-\t\tsize_t offset = i-\u003eiov_offset, fsize = folioq_folio_size(folioq, slot);\n-\t\tsize_t part = PAGE_SIZE - offset % PAGE_SIZE;\n-\n-\t\tif (offset \u003c fsize) {\n-\t\t\tpart = umin(part, umin(maxsize - extracted, fsize - offset));\n-\t\t\ti-\u003ecount -= part;\n-\t\t\ti-\u003eiov_offset += part;\n-\t\t\textracted += part;\n+\t/* Now transcribe the page pointers. */\n+\textracted = 0;\n+\tbvecq = iter-\u003ebvecq;\n+\toffset = iter-\u003eiov_offset;\n+\tslot = iter-\u003ebvecq_slot;\n+\n+\tdo {\n+\t\tconst struct bio_vec *bv;\n+\t\tsize_t boff, blen;\n \n-\t\t\tp[nr++] = folio_page(folio, offset / PAGE_SIZE);\n+\t\tif (slot \u003e= bvecq-\u003enr_slots) {\n+\t\t\tif (!bvecq-\u003enext) {\n+\t\t\t\tWARN_ON_ONCE(extracted \u003c iter-\u003ecount);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tbvecq = bvecq-\u003enext;\n+\t\t\tslot = 0;\n+\t\t\toffset = 0;\n+\t\t\tcontinue;\n \t\t}\n \n-\t\tif (nr \u003e= maxpages || extracted \u003e= maxsize)\n-\t\t\tbreak;\n+\t\tbv = \u0026bvecq-\u003ebv[slot];\n+\t\tboff = bv-\u003ebv_offset;\n+\t\tblen = bv-\u003ebv_len;\n \n-\t\tif (i-\u003eiov_offset \u003e= fsize) {\n-\t\t\ti-\u003eiov_offset = 0;\n+\t\t/* bv_page is not allowed to be NULL unless bv_len == 0. */\n+\n+\t\tif (offset \u003c blen) {\n+\t\t\tsize_t part = umin(maxsize - extracted, blen - offset);\n+\t\t\tsize_t poff = (boff + offset) % PAGE_SIZE;\n+\t\t\tsize_t pix = (boff + offset) / PAGE_SIZE;\n+\n+\t\t\tif (poff + part \u003e PAGE_SIZE)\n+\t\t\t\tpart = PAGE_SIZE - poff;\n+\n+\t\t\tif (!extracted)\n+\t\t\t\t*offset0 = poff;\n+\n+\t\t\tp[nr++] = bv-\u003ebv_page + pix;\n+\t\t\toffset += part;\n+\t\t\textracted += part;\n+\t\t}\n+\n+\t\tif (offset \u003e= blen) {\n+\t\t\toffset = 0;\n \t\t\tslot++;\n-\t\t\tif (slot == folioq_nr_slots(folioq) \u0026\u0026 folioq-\u003enext) {\n-\t\t\t\tfolioq = folioq-\u003enext;\n-\t\t\t\tslot = 0;\n-\t\t\t}\n \t\t}\n-\t}\n+\t} while (nr \u003c maxpages \u0026\u0026 extracted \u003c maxsize);\n \n-\ti-\u003efolioq = folioq;\n-\ti-\u003efolioq_slot = slot;\n+\titer-\u003ebvecq = bvecq;\n+\titer-\u003ebvecq_slot = slot;\n+\titer-\u003eiov_offset = offset;\n+\titer-\u003ecount -= extracted;\n \treturn extracted;\n }\n \n@@ -1817,8 +2015,8 @@ static ssize_t iov_iter_extract_user_pages(struct iov_iter *i,\n * added to the pages, but refs will not be taken.\n * iov_iter_extract_will_pin() will return true.\n *\n- * (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_FOLIOQ or ITER_XARRAY, the\n- * pages are merely listed; no extra refs or pins are obtained.\n+ * (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_XARRAY, the pages are\n+ * merely listed; no extra refs or pins are obtained.\n * iov_iter_extract_will_pin() will return 0.\n *\n * Note also:\n@@ -1853,10 +2051,10 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,\n \t\treturn iov_iter_extract_bvec_pages(i, pages, maxsize,\n \t\t\t\t\t\t maxpages, extraction_flags,\n \t\t\t\t\t\t offset0);\n-\tif (iov_iter_is_folioq(i))\n-\t\treturn iov_iter_extract_folioq_pages(i, pages, maxsize,\n-\t\t\t\t\t\t maxpages, extraction_flags,\n-\t\t\t\t\t\t offset0);\n+\tif (iov_iter_is_bvecq(i))\n+\t\treturn iov_iter_extract_bvecq_pages(i, pages, maxsize,\n+\t\t\t\t\t\t maxpages, extraction_flags,\n+\t\t\t\t\t\t offset0);\n \tif (iov_iter_is_xarray(i))\n \t\treturn iov_iter_extract_xarray_pages(i, pages, maxsize,\n \t\t\t\t\t\t maxpages, extraction_flags,\ndiff --git a/lib/scatterlist.c b/lib/scatterlist.c\nindex 6ea40d2e62475..b9a7298306d95 100644\n--- a/lib/scatterlist.c\n+++ b/lib/scatterlist.c\n@@ -10,8 +10,8 @@\n #include \u003clinux/highmem.h\u003e\n #include \u003clinux/kmemleak.h\u003e\n #include \u003clinux/bvec.h\u003e\n+#include \u003clinux/bvecq.h\u003e\n #include \u003clinux/uio.h\u003e\n-#include \u003clinux/folio_queue.h\u003e\n \n /**\n * sg_nents - return total count of entries in scatterlist\n@@ -1268,61 +1268,59 @@ static ssize_t extract_kvec_to_sg(struct iov_iter *iter,\n }\n \n /*\n- * Extract up to sg_max folios from an FOLIOQ-type iterator and add them to\n+ * Extract up to sg_max folios from an BVECQ-type iterator and add them to\n * the scatterlist. The pages are not pinned.\n */\n-static ssize_t extract_folioq_to_sg(struct iov_iter *iter,\n+static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,\n \t\t\t\t ssize_t maxsize,\n \t\t\t\t struct sg_table *sgtable,\n \t\t\t\t unsigned int sg_max,\n \t\t\t\t iov_iter_extraction_t extraction_flags)\n {\n-\tconst struct folio_queue *folioq = iter-\u003efolioq;\n+\tconst struct bvecq *bvecq = iter-\u003ebvecq;\n \tstruct scatterlist *sg = sgtable-\u003esgl + sgtable-\u003enents;\n-\tunsigned int slot = iter-\u003efolioq_slot;\n+\tunsigned int slot = iter-\u003ebvecq_slot;\n \tssize_t ret = 0;\n \tsize_t offset = iter-\u003eiov_offset;\n \n-\tBUG_ON(!folioq);\n+\tmaxsize = umin(maxsize, iov_iter_count(iter));\n \n-\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\tfolioq = folioq-\u003enext;\n-\t\tif (WARN_ON_ONCE(!folioq))\n-\t\t\treturn 0;\n-\t\tslot = 0;\n-\t}\n-\n-\tdo {\n-\t\tstruct folio *folio = folioq_folio(folioq, slot);\n-\t\tsize_t fsize = folioq_folio_size(folioq, slot);\n+\twhile (sg_max \u003e 0 \u0026\u0026 ret \u003c maxsize) {\n+\t\tconst struct bio_vec *bv;\n+\t\tsize_t blen, part;\n \n-\t\tif (offset \u003c fsize) {\n-\t\t\tsize_t part = umin(maxsize - ret, fsize - offset);\n-\n-\t\t\tsg_set_page(sg, folio_page(folio, 0), part, offset);\n-\t\t\tsgtable-\u003enents++;\n-\t\t\tsg++;\n-\t\t\tsg_max--;\n-\t\t\toffset += part;\n-\t\t\tret += part;\n+\t\tif (slot \u003e= bvecq-\u003enr_slots) {\n+\t\t\tif (!bvecq-\u003enext) {\n+\t\t\t\tWARN_ON_ONCE(ret \u003c iter-\u003ecount);\n+\t\t\t\tbreak;\n+\t\t\t}\n+\t\t\tbvecq = bvecq-\u003enext;\n+\t\t\tslot = 0;\n+\t\t\toffset = 0;\n+\t\t\tcontinue;\n \t\t}\n \n-\t\tif (offset \u003e= fsize) {\n+\t\tbv = \u0026bvecq-\u003ebv[slot];\n+\t\tblen = bv-\u003ebv_len;\n+\n+\t\tif (offset \u003e= blen) {\n \t\t\toffset = 0;\n \t\t\tslot++;\n-\t\t\tif (slot \u003e= folioq_nr_slots(folioq)) {\n-\t\t\t\tif (!folioq-\u003enext) {\n-\t\t\t\t\tWARN_ON_ONCE(ret \u003c iter-\u003ecount);\n-\t\t\t\t\tbreak;\n-\t\t\t\t}\n-\t\t\t\tfolioq = folioq-\u003enext;\n-\t\t\t\tslot = 0;\n-\t\t\t}\n+\t\t\tcontinue;\n \t\t}\n-\t} while (sg_max \u003e 0 \u0026\u0026 ret \u003c maxsize);\n \n-\titer-\u003efolioq = folioq;\n-\titer-\u003efolioq_slot = slot;\n+\t\tpart = umin(maxsize - ret, blen - offset);\n+\n+\t\tsg_set_page(sg, bv-\u003ebv_page, part, bv-\u003ebv_offset + offset);\n+\t\tsgtable-\u003enents++;\n+\t\tsg++;\n+\t\tsg_max--;\n+\t\toffset += part;\n+\t\tret += part;\n+\t}\n+\n+\titer-\u003ebvecq = bvecq;\n+\titer-\u003ebvecq_slot = slot;\n \titer-\u003eiov_offset = offset;\n \titer-\u003ecount -= ret;\n \treturn ret;\n@@ -1391,8 +1389,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,\n * addition of @sg_max elements.\n *\n * The pages referred to by UBUF- and IOVEC-type iterators are extracted and\n- * pinned; BVEC-, KVEC-, FOLIOQ- and XARRAY-type are extracted but aren't\n- * pinned; DISCARD-type is not supported.\n+ * pinned; BVEC-, BVECQ-, KVEC-, XARRAY-type are extracted but aren't pinned;\n+ * DISCARD-type is not supported.\n *\n * No end mark is placed on the scatterlist; that's left to the caller.\n *\n@@ -1424,9 +1422,9 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,\n \tcase ITER_KVEC:\n \t\treturn extract_kvec_to_sg(iter, maxsize, sgtable, sg_max,\n \t\t\t\t\t extraction_flags);\n-\tcase ITER_FOLIOQ:\n-\t\treturn extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,\n-\t\t\t\t\t extraction_flags);\n+\tcase ITER_BVECQ:\n+\t\treturn extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,\n+\t\t\t\t\t extraction_flags);\n \tcase ITER_XARRAY:\n \t\treturn extract_xarray_to_sg(iter, maxsize, sgtable, sg_max,\n \t\t\t\t\t extraction_flags);\ndiff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c\nindex 32e42d8c7ca13..a2a83ba613fda 100644\n--- a/lib/tests/kunit_iov_iter.c\n+++ b/lib/tests/kunit_iov_iter.c\n@@ -12,7 +12,7 @@\n #include \u003clinux/mm.h\u003e\n #include \u003clinux/uio.h\u003e\n #include \u003clinux/bvec.h\u003e\n-#include \u003clinux/folio_queue.h\u003e\n+#include \u003clinux/bvecq.h\u003e\n #include \u003clinux/scatterlist.h\u003e\n #include \u003clinux/minmax.h\u003e\n #include \u003clinux/mman.h\u003e\n@@ -382,58 +382,67 @@ static void __init iov_kunit_copy_from_bvec(struct kunit *test)\n \tKUNIT_SUCCEED(test);\n }\n \n-static void iov_kunit_destroy_folioq(void *data)\n+static void iov_kunit_destroy_bvecq(void *data)\n {\n-\tstruct folio_queue *folioq, *next;\n+\tstruct bvecq *bq, *next;\n \n-\tfor (folioq = data; folioq; folioq = next) {\n-\t\tnext = folioq-\u003enext;\n-\t\tkfree(folioq);\n+\tfor (bq = data; bq; bq = next) {\n+\t\tnext = bq-\u003enext;\n+\t\t/* The pages are freed by vmap with VM_MAP_PUT_PAGES. */\n+\t\tkfree(bq);\n \t}\n }\n \n-static void __init iov_kunit_load_folioq(struct kunit *test,\n+static struct bvecq *iov_kunit_alloc_bvecq(struct kunit *test, unsigned int max_slots)\n+{\n+\tstruct bvecq *bq;\n+\n+\tbq = kzalloc(struct_size(bq, __bv, max_slots), GFP_KERNEL);\n+\tKUNIT_ASSERT_NOT_ERR_OR_NULL(test, bq);\n+\tbq-\u003emax_slots = max_slots;\n+\tbq-\u003ebv = bq-\u003e__bv;\n+\tbq-\u003einline_bv = true;\n+\treturn bq;\n+}\n+\n+static struct bvecq *iov_kunit_create_bvecq(struct kunit *test, unsigned int max_slots)\n+{\n+\tstruct bvecq *bq;\n+\n+\tbq = iov_kunit_alloc_bvecq(test, max_slots);\n+\tkunit_add_action_or_reset(test, iov_kunit_destroy_bvecq, bq);\n+\treturn bq;\n+}\n+\n+static void __init iov_kunit_load_bvecq(struct kunit *test,\n \t\t\t\t\tstruct iov_iter *iter, int dir,\n-\t\t\t\t\tstruct folio_queue *folioq,\n+\t\t\t\t\tstruct bvecq *bq_head,\n \t\t\t\t\tstruct page **pages, size_t npages)\n {\n-\tstruct folio_queue *p = folioq;\n+\tstruct bvecq *bq = bq_head;\n \tsize_t size = 0;\n-\tint i;\n \n-\tfor (i = 0; i \u003c npages; i++) {\n-\t\tif (folioq_full(p)) {\n-\t\t\tp-\u003enext = kzalloc_obj(struct folio_queue);\n-\t\t\tKUNIT_ASSERT_NOT_ERR_OR_NULL(test, p-\u003enext);\n-\t\t\tfolioq_init(p-\u003enext, 0);\n-\t\t\tp-\u003enext-\u003eprev = p;\n-\t\t\tp = p-\u003enext;\n+\tfor (int i = 0; i \u003c npages; i++) {\n+\t\tif (bq-\u003enr_slots \u003e= bq-\u003emax_slots) {\n+\t\t\tbq-\u003enext = iov_kunit_alloc_bvecq(test, 13);\n+\t\t\tbq-\u003enext-\u003eprev = bq;\n+\t\t\tbq = bq-\u003enext;\n \t\t}\n-\t\tfolioq_append(p, page_folio(pages[i]));\n+\t\tbvec_set_page(\u0026bq-\u003ebv[bq-\u003enr_slots], pages[i], PAGE_SIZE, 0);\n+\t\tbq-\u003enr_slots++;\n \t\tsize += PAGE_SIZE;\n \t}\n-\tiov_iter_folio_queue(iter, dir, folioq, 0, 0, size);\n-}\n-\n-static struct folio_queue *iov_kunit_create_folioq(struct kunit *test)\n-{\n-\tstruct folio_queue *folioq;\n-\n-\tfolioq = kzalloc_obj(struct folio_queue);\n-\tKUNIT_ASSERT_NOT_ERR_OR_NULL(test, folioq);\n-\tkunit_add_action_or_reset(test, iov_kunit_destroy_folioq, folioq);\n-\tfolioq_init(folioq, 0);\n-\treturn folioq;\n+\tiov_iter_bvec_queue(iter, dir, bq_head, 0, 0, size);\n }\n \n /*\n- * Test copying to a ITER_FOLIOQ-type iterator.\n+ * Test copying to a ITER_BVECQ-type iterator.\n */\n-static void __init iov_kunit_copy_to_folioq(struct kunit *test)\n+static void __init iov_kunit_copy_to_bvecq(struct kunit *test)\n {\n \tconst struct kvec_test_range *pr;\n \tstruct iov_iter iter;\n-\tstruct folio_queue *folioq;\n+\tstruct bvecq *bq;\n \tstruct page **spages, **bpages;\n \tu8 *scratch, *buffer;\n \tsize_t bufsize, npages, size, copied;\n@@ -442,7 +451,7 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)\n \tbufsize = 0x100000;\n \tnpages = bufsize / PAGE_SIZE;\n \n-\tfolioq = iov_kunit_create_folioq(test);\n+\tbq = iov_kunit_create_bvecq(test, 13);\n \n \tscratch = iov_kunit_create_buffer(test, \u0026spages, npages);\n \tfor (i = 0; i \u003c bufsize; i++)\n@@ -451,20 +460,19 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)\n \tbuffer = iov_kunit_create_buffer(test, \u0026bpages, npages);\n \tmemset(buffer, 0, bufsize);\n \n-\tiov_kunit_load_folioq(test, \u0026iter, READ, folioq, bpages, npages);\n+\tiov_kunit_load_bvecq(test, \u0026iter, READ, bq, bpages, npages);\n \n \ti = 0;\n \tfor (pr = kvec_test_ranges; pr-\u003efrom \u003e= 0; pr++) {\n \t\tsize = pr-\u003eto - pr-\u003efrom;\n \t\tKUNIT_ASSERT_LE(test, pr-\u003eto, bufsize);\n \n-\t\tiov_iter_folio_queue(\u0026iter, READ, folioq, 0, 0, pr-\u003eto);\n+\t\tiov_iter_bvec_queue(\u0026iter, READ, bq, 0, 0, pr-\u003eto);\n \t\tiov_iter_advance(\u0026iter, pr-\u003efrom);\n \t\tcopied = copy_to_iter(scratch + i, size, \u0026iter);\n \n \t\tKUNIT_EXPECT_EQ(test, copied, size);\n \t\tKUNIT_EXPECT_EQ(test, iter.count, 0);\n-\t\tKUNIT_EXPECT_EQ(test, iter.iov_offset, pr-\u003eto % PAGE_SIZE);\n \t\ti += size;\n \t\tif (test-\u003estatus == KUNIT_FAILURE)\n \t\t\tgoto stop;\n@@ -489,13 +497,13 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)\n }\n \n /*\n- * Test copying from a ITER_FOLIOQ-type iterator.\n+ * Test copying from a ITER_BVECQ-type iterator.\n */\n-static void __init iov_kunit_copy_from_folioq(struct kunit *test)\n+static void __init iov_kunit_copy_from_bvecq(struct kunit *test)\n {\n \tconst struct kvec_test_range *pr;\n \tstruct iov_iter iter;\n-\tstruct folio_queue *folioq;\n+\tstruct bvecq *bq;\n \tstruct page **spages, **bpages;\n \tu8 *scratch, *buffer;\n \tsize_t bufsize, npages, size, copied;\n@@ -504,7 +512,7 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)\n \tbufsize = 0x100000;\n \tnpages = bufsize / PAGE_SIZE;\n \n-\tfolioq = iov_kunit_create_folioq(test);\n+\tbq = iov_kunit_create_bvecq(test, 13);\n \n \tbuffer = iov_kunit_create_buffer(test, \u0026bpages, npages);\n \tfor (i = 0; i \u003c bufsize; i++)\n@@ -513,20 +521,19 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)\n \tscratch = iov_kunit_create_buffer(test, \u0026spages, npages);\n \tmemset(scratch, 0, bufsize);\n \n-\tiov_kunit_load_folioq(test, \u0026iter, READ, folioq, bpages, npages);\n+\tiov_kunit_load_bvecq(test, \u0026iter, READ, bq, bpages, npages);\n \n \ti = 0;\n \tfor (pr = kvec_test_ranges; pr-\u003efrom \u003e= 0; pr++) {\n \t\tsize = pr-\u003eto - pr-\u003efrom;\n \t\tKUNIT_ASSERT_LE(test, pr-\u003eto, bufsize);\n \n-\t\tiov_iter_folio_queue(\u0026iter, WRITE, folioq, 0, 0, pr-\u003eto);\n+\t\tiov_iter_bvec_queue(\u0026iter, WRITE, bq, 0, 0, pr-\u003eto);\n \t\tiov_iter_advance(\u0026iter, pr-\u003efrom);\n \t\tcopied = copy_from_iter(scratch + i, size, \u0026iter);\n \n \t\tKUNIT_EXPECT_EQ(test, copied, size);\n \t\tKUNIT_EXPECT_EQ(test, iter.count, 0);\n-\t\tKUNIT_EXPECT_EQ(test, iter.iov_offset, pr-\u003eto % PAGE_SIZE);\n \t\ti += size;\n \t}\n \n@@ -868,13 +875,13 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)\n }\n \n /*\n- * Test the extraction of ITER_FOLIOQ-type iterators.\n+ * Test the extraction of ITER_BVECQ-type iterators.\n */\n-static void __init iov_kunit_extract_pages_folioq(struct kunit *test)\n+static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)\n {\n \tconst struct kvec_test_range *pr;\n-\tstruct folio_queue *folioq;\n \tstruct iov_iter iter;\n+\tstruct bvecq *bq;\n \tstruct page **bpages, *pagelist[8], **pages = pagelist;\n \tssize_t len;\n \tsize_t bufsize, size = 0, npages;\n@@ -883,17 +890,17 @@ static void __init iov_kunit_extract_pages_folioq(struct kunit *test)\n \tbufsize = 0x100000;\n \tnpages = bufsize / PAGE_SIZE;\n \n-\tfolioq = iov_kunit_create_folioq(test);\n+\tbq = iov_kunit_create_bvecq(test, 13);\n \n \tiov_kunit_create_buffer(test, \u0026bpages, npages);\n-\tiov_kunit_load_folioq(test, \u0026iter, READ, folioq, bpages, npages);\n+\tiov_kunit_load_bvecq(test, \u0026iter, READ, bq, bpages, npages);\n \n \tfor (pr = kvec_test_ranges; pr-\u003efrom \u003e= 0; pr++) {\n \t\tfrom = pr-\u003efrom;\n \t\tsize = pr-\u003eto - from;\n \t\tKUNIT_ASSERT_LE(test, pr-\u003eto, bufsize);\n \n-\t\tiov_iter_folio_queue(\u0026iter, WRITE, folioq, 0, 0, pr-\u003eto);\n+\t\tiov_iter_bvec_queue(\u0026iter, WRITE, bq, 0, 0, pr-\u003eto);\n \t\tiov_iter_advance(\u0026iter, from);\n \n \t\tdo {\n@@ -1173,23 +1180,6 @@ static void __init iov_kunit_iter_to_sg_bvec(struct kunit *test)\n \tiov_kunit_iter_to_sg_check(test, \u0026iter, bufsize, \u0026data);\n }\n \n-static void __init iov_kunit_iter_to_sg_folioq(struct kunit *test)\n-{\n-\tstruct iov_kunit_iter_to_sg_data data;\n-\tstruct folio_queue *folioq;\n-\tstruct iov_iter iter;\n-\tsize_t bufsize;\n-\n-\tbufsize = 0x200000;\n-\tiov_kunit_iter_to_sg_init(test, bufsize, false, \u0026data);\n-\n-\tfolioq = iov_kunit_create_folioq(test);\n-\tiov_kunit_load_folioq(test, \u0026iter, READ, folioq, data.pages,\n-\t\t\t data.npages);\n-\n-\tiov_kunit_iter_to_sg_check(test, \u0026iter, bufsize, \u0026data);\n-}\n-\n static void __init iov_kunit_iter_to_sg_xarray(struct kunit *test)\n {\n \tstruct iov_kunit_iter_to_sg_data data;\n@@ -1226,17 +1216,16 @@ static struct kunit_case __refdata iov_kunit_cases[] = {\n \tKUNIT_CASE(iov_kunit_copy_from_kvec),\n \tKUNIT_CASE(iov_kunit_copy_to_bvec),\n \tKUNIT_CASE(iov_kunit_copy_from_bvec),\n-\tKUNIT_CASE(iov_kunit_copy_to_folioq),\n-\tKUNIT_CASE(iov_kunit_copy_from_folioq),\n+\tKUNIT_CASE(iov_kunit_copy_to_bvecq),\n+\tKUNIT_CASE(iov_kunit_copy_from_bvecq),\n \tKUNIT_CASE(iov_kunit_copy_to_xarray),\n \tKUNIT_CASE(iov_kunit_copy_from_xarray),\n \tKUNIT_CASE(iov_kunit_extract_pages_kvec),\n \tKUNIT_CASE(iov_kunit_extract_pages_bvec),\n-\tKUNIT_CASE(iov_kunit_extract_pages_folioq),\n+\tKUNIT_CASE(iov_kunit_extract_pages_bvecq),\n \tKUNIT_CASE(iov_kunit_extract_pages_xarray),\n \tKUNIT_CASE(iov_kunit_iter_to_sg_kvec),\n \tKUNIT_CASE(iov_kunit_iter_to_sg_bvec),\n-\tKUNIT_CASE(iov_kunit_iter_to_sg_folioq),\n \tKUNIT_CASE(iov_kunit_iter_to_sg_xarray),\n \tKUNIT_CASE(iov_kunit_iter_to_sg_ubuf),\n \t{}\n"
} |
| 2/1 | 2026/09/30 13:44 | agent | patch-evaluator |
3mModel:core Args: null Results: {
"EnableConfigs": [
"NETFS_SUPPORT",
"AFS_FS",
"CIFS"
],
"FocusSymbols": [
"bvecq_alloc_one",
"bvecq_alloc_chain",
"bvecq_alloc_buffer2",
"bvecq_put",
"bvecq_expand_buffer",
"iov_iter_bvec_queue",
"rolling_buffer_init",
"rolling_buffer_make_space",
"rolling_buffer_bulk_load_from_ra",
"rolling_buffer_append",
"rolling_buffer_delete_spent",
"netfs_writeback_single",
"afs_dir_find_block",
"afs_read_dir"
],
"Reasoning": "The patch refactors the kernel's segmented folio queue (`folio_queue` / `ITER_FOLIOQ`) into a segmented bio_vec queue (`bvecq` / `ITER_BVECQ`). It introduces a new `fs/netfs/bvecq.c` implementation with custom memory management (including bulk page allocations overlaid with bio_vec structs), lockless queue manipulations using acquire/release barriers, replaces iterator iteration and extraction algorithms in `lib/iov_iter.c` and `lib/scatterlist.c`, updates netfs rolling buffers, and refactors callers in AFS and SMB/CIFS. The newly introduced and modified code runs in reachable core subsystem and filesystem paths and adds several defensive WARN_ON assertions, warranting fuzzing.",
"WorthFuzzing": true
}Instruction: You are an expert Linux kernel maintainer and security engineer.
Your job is to review a provided patch series and evaluate whether it warrants fuzzing with syzkaller.
IMPORTANT: The changes have ALREADY been applied and committed as the HEAD commit in
your workspace. Do NOT rely on internal assumptions. You must actively use your code access
tools to inspect the actual source code, callers, and surrounding context.
================================================================================
1. CORE TRIAGE PHILOSOPHY
================================================================================
The goal of patch fuzzing is to discover crashes, regressions, exposed latent bugs,
and newly triggered assertions introduced by the patch series.
- REACHABILITY IS THE PRIMARY GATE:
Fuzzing can only discover bugs in code that can actually execute in standard virtualized
environments (GCE or QEMU, utilizing software-emulated devices like USB gadgets, netdev, tun/tap).
If the modified code is structurally unreachable (see Section 2), it MUST NOT be fuzzed,
regardless of whether it adds assertions or complex logic.
- DO NOT BLINDLY TRUST "NO FUNCTIONAL CHANGE" (NFCI) OR "REFACTORING" CLAIMS:
Patch authors routinely label changes as "cleanups", "refactorings", or state
"No functional change intended". Do NOT take these claims at face value.
Code refactorings that rearrange logic, introduce helper functions, or alter state management
in core subsystems frequently introduce subtle semantic shifts or uncover latent kernel bugs.
If reachable executable code is modified or refactored, it MUST be fuzzed.
- NEW OR MODIFIED ASSERTIONS IN REACHABLE CODE MUST BE FUZZED:
When a patch introduces or modifies runtime checks or assertions (e.g., WARN_ON*, VM_WARN_ON*,
BUG_ON*, lockdep_assert*) in reachable code paths, it enforces new or stricter invariants.
Even if the author believes the invariant always holds, fuzzing is essential to verify whether
an unusual sequence of operations can violate it.
================================================================================
2. WHEN TO RETURN WorthFuzzing=false (NEGATIVE CRITERIA)
================================================================================
Return WorthFuzzing=false ONLY IF all modified code falls strictly into one or more of these categories:
- Non-kernel and non-executable changes:
* Modifications to Documentation/, comments, or spelling fixes.
* User-space directories, self-tests, samples, or scripts (e.g., tools/, samples/, scripts/, usr/)
that do not affect the compiled kernel image (vmlinux) or kernel modules.
* Purely decorative logging (e.g., message strings in pr_err, printk, dev_info) or tracepoints
that do not alter control flow or data structures.
* Build system or Kconfig changes that do not alter compiled C logic.
- Structurally unreachable hardware:
* Vendor-specific PCIe switches, SmartNICs, or GPU drivers (e.g., mlxsw, pds_core, qed,
ionic, amdgpu) requiring physical ASIC/PCIe cards not emulated in standard QEMU.
- Unreachable execution paths:
* Driver teardown callbacks (.remove, .shutdown, pci_unregister_driver) executed only during
physical PCI hot-unplug or manual sysfs driver unbinding.
* Code paths exclusive to architectures other than the target architecture.
================================================================================
3. WHEN TO RETURN WorthFuzzing=true (POSITIVE CRITERIA)
================================================================================
Return WorthFuzzing=true whenever the patch touches reachable executable code, including:
- Core Subsystems:
* Any logic modifications in memory management (mm/), synchronization/locking (kernel/locking/),
BPF, scheduler, core networking, VFS, or syscall handling.
- Refactorings and Code Cleanups:
* Any restructuring of reachable data structures, helper abstractions, or algorithm flows.
- Runtime Assertions and Defensive Checks:
* Any introduction or alteration of assertions (WARN_ON*, VM_WARN_ON*, BUG_ON*, etc.) in reachable paths.
- Reachable Drivers and Protocols:
* Drivers accessible via virtual buses (virtio, USB gadget, loopback, netlink, binder, sockets, etc.).
================================================================================
4. EXTRACTING FocusSymbols (PREVENTING DILUTION)
================================================================================
When WorthFuzzing=true, you must extract specific kernel functions into FocusSymbols to guide the fuzzer:
- AVOID UBIQUITOUS LIFECYCLE HOT-PATHS:
Do NOT list generic, ubiquitous functions called by almost every program in the corpus
(including, but not limited to: general memory allocators and deallocators, page fault
and trap handlers, or core synchronization primitives; this is not an exhaustive list).
Listing ubiquitous functions causes the fuzzer to classify thousands of unrelated tests as "focused",
which severely dilutes fuzzing effort away from the actual changes.
- TARGET SPECIFIC FEATURE LOGIC AND ENTRYPOINTS:
List functions that specifically implement the logic being added or altered, or direct API entrypoints
for the subsystem feature under review.
- HANDLING STATIC INLINE FUNCTIONS IN HEADERS (.h):
Compiler-inlined static functions (such as static inlines in mm/*.h or include/linux/*.h) lack
distinct symbol addresses in vmlinux and cannot be targeted directly by symbol coverage filters.
If the changes are primarily in static inline helpers, identify non-static, feature-specific caller
functions in .c files that exercise them (avoiding ubiquitous lifecycle wrappers).
================================================================================
5. IDENTIFYING EnableConfigs
================================================================================
Identify any specific CONFIG_ options required to properly compile and reach the modified code:
- Inspect Kconfig files and #ifdef guards; do not make assumptions.
- Check "depends on" lines in Kconfig to include any non-standard parent subsystem configs needed.
- Strip any 'CONFIG_' prefix (e.g., return "NET_IPV4" instead of "CONFIG_NET_IPV4").
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit e4563b15c78bd5d23f58d5727c178d8ded28c31b
Author: syz-cluster <triage@syzkaller.com>
Date: Wed Sep 30 13:44:48 2026 +0000
syz-cluster: applied patch under review
diff --git a/Documentation/core-api/folio_queue.rst b/Documentation/core-api/folio_queue.rst
deleted file mode 100644
index b7628896d2b69..0000000000000
--- a/Documentation/core-api/folio_queue.rst
+++ /dev/null
@@ -1,209 +0,0 @@
-.. SPDX-License-Identifier: GPL-2.0+
-
-===========
-Folio Queue
-===========
-
-:Author: David Howells <dhowells@redhat.com>
-
-.. Contents:
-
- * Overview
- * Initialisation
- * Adding and removing folios
- * Querying information about a folio
- * Querying information about a folio_queue
- * Folio queue iteration
- * Folio marks
- * Lockless simultaneous production/consumption issues
-
-
-Overview
-========
-
-The folio_queue struct forms a single segment in a segmented list of folios
-that can be used to form an I/O buffer. As such, the list can be iterated over
-using the ITER_FOLIOQ iov_iter type.
-
-The publicly accessible members of the structure are::
-
- struct folio_queue {
- struct folio_queue *next;
- struct folio_queue *prev;
- ...
- };
-
-A pair of pointers are provided, ``next`` and ``prev``, that point to the
-segments on either side of the segment being accessed. Whilst this is a
-doubly-linked list, it is intentionally not a circular list; the outward
-sibling pointers in terminal segments should be NULL.
-
-Each segment in the list also stores:
-
- * an ordered sequence of folio pointers,
- * the size of each folio and
- * three 1-bit marks per folio,
-
-but these should not be accessed directly as the underlying data structure may
-change, but rather the access functions outlined below should be used.
-
-The facility can be made accessible by::
-
- #include <linux/folio_queue.h>
-
-and to use the iterator::
-
- #include <linux/uio.h>
-
-
-Initialisation
-==============
-
-A segment should be initialised by calling::
-
- void folioq_init(struct folio_queue *folioq);
-
-with a pointer to the segment to be initialised. Note that this will not
-necessarily initialise all the folio pointers, so care must be taken to check
-the number of folios added.
-
-
-Adding and removing folios
-==========================
-
-Folios can be set in the next unused slot in a segment struct by calling one
-of::
-
- unsigned int folioq_append(struct folio_queue *folioq,
- struct folio *folio);
-
- unsigned int folioq_append_mark(struct folio_queue *folioq,
- struct folio *folio);
-
-Both functions update the stored folio count, store the folio and note its
-size. The second function also sets the first mark for the folio added. Both
-functions return the number of the slot used. [!] Note that no attempt is made
-to check that the capacity wasn't overrun and the list will not be extended
-automatically.
-
-A folio can be excised by calling::
-
- void folioq_clear(struct folio_queue *folioq, unsigned int slot);
-
-This clears the slot in the array and also clears all the marks for that folio,
-but doesn't change the folio count - so future accesses of that slot must check
-if the slot is occupied.
-
-
-Querying information about a folio
-==================================
-
-Information about the folio in a particular slot may be queried by the
-following function::
-
- struct folio *folioq_folio(const struct folio_queue *folioq,
- unsigned int slot);
-
-If a folio has not yet been set in that slot, this may yield an undefined
-pointer. The size of the folio in a slot may be queried with either of::
-
- unsigned int folioq_folio_order(const struct folio_queue *folioq,
- unsigned int slot);
-
- size_t folioq_folio_size(const struct folio_queue *folioq,
- unsigned int slot);
-
-The first function returns the size as an order and the second as a number of
-bytes.
-
-
-Querying information about a folio_queue
-========================================
-
-Information may be retrieved about a particular segment with the following
-functions::
-
- unsigned int folioq_nr_slots(const struct folio_queue *folioq);
-
- unsigned int folioq_count(struct folio_queue *folioq);
-
- bool folioq_full(struct folio_queue *folioq);
-
-The first function returns the maximum capacity of a segment. It must not be
-assumed that this won't vary between segments. The second returns the number
-of folios added to a segments and the third is a shorthand to indicate if the
-segment has been filled to capacity.
-
-Not that the count and fullness are not affected by clearing folios from the
-segment. These are more about indicating how many slots in the array have been
-initialised, and it assumed that slots won't get reused, but rather the segment
-will get discarded as the queue is consumed.
-
-
-Folio marks
-===========
-
-Folios within a queue can also have marks assigned to them. These marks can be
-used to note information such as if a folio needs folio_put() calling upon it.
-There are three marks available to be set for each folio.
-
-The marks can be set by::
-
- void folioq_mark(struct folio_queue *folioq, unsigned int slot);
- void folioq_mark2(struct folio_queue *folioq, unsigned int slot);
-
-Cleared by::
-
- void folioq_unmark(struct folio_queue *folioq, unsigned int slot);
- void folioq_unmark2(struct folio_queue *folioq, unsigned int slot);
-
-And the marks can be queried by::
-
- bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot);
- bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot);
-
-The marks can be used for any purpose and are not interpreted by this API.
-
-
-Folio queue iteration
-=====================
-
-A list of segments may be iterated over using the I/O iterator facility using
-an ``iov_iter`` iterator of ``ITER_FOLIOQ`` type. The iterator may be
-initialised with::
-
- void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
- const struct folio_queue *folioq,
- unsigned int first_slot, unsigned int offset,
- size_t count);
-
-This may be told to start at a particular segment, slot and offset within a
-queue. The iov iterator functions will follow the next pointers when advancing
-and prev pointers when reverting when needed.
-
-
-Lockless simultaneous production/consumption issues
-===================================================
-
-If properly managed, the list can be extended by the producer at the head end
-and shortened by the consumer at the tail end simultaneously without the need
-to take locks. The ITER_FOLIOQ iterator inserts appropriate barriers to aid
-with this.
-
-Care must be taken when simultaneously producing and consuming a list. If the
-last segment is reached and the folios it refers to are entirely consumed by
-the IOV iterators, an iov_iter struct will be left pointing to the last segment
-with a slot number equal to the capacity of that segment. The iterator will
-try to continue on from this if there's another segment available when it is
-used again, but care must be taken lest the segment got removed and freed by
-the consumer before the iterator was advanced.
-
-It is recommended that the queue always contain at least one segment, even if
-that segment has never been filled or is entirely spent. This prevents the
-head and tail pointers from collapsing.
-
-
-API Function Reference
-======================
-
-.. kernel-doc:: include/linux/folio_queue.h
diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
index 92f91c6a0d79d..956f6803d2a22 100644
--- a/Documentation/core-api/index.rst
+++ b/Documentation/core-api/index.rst
@@ -39,7 +39,6 @@ Library functionality that is used throughout the kernel.
kref
cleanup
assoc_array
- folio_queue
xarray
maple_tree
idr
diff --git a/Documentation/filesystems/netfs_library.rst b/Documentation/filesystems/netfs_library.rst
index 0c9786ffe1924..ebf07a37922e8 100644
--- a/Documentation/filesystems/netfs_library.rst
+++ b/Documentation/filesystems/netfs_library.rst
@@ -452,7 +452,7 @@ be called from the writeback code to write the data to the cache, if there is
one.
The inode should be marked ``NETFS_ICTX_SINGLE_NO_UPLOAD`` if this API is to be
-used. The writeback function requires the buffer to be of ITER_FOLIOQ type.
+used.
High-Level VM API
==================
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 2db534a2c7cc4..5fac02d2d2814 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -140,9 +140,9 @@ static void afs_dir_dump(struct afs_vnode *dvnode)
pr_warn("DIR %llx:%llx is=%llx\n",
dvnode->fid.vid, dvnode->fid.vnode, i_size);
- iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
- iterate_folioq(&iter, iov_iter_count(&iter), NULL, NULL,
- afs_dir_dump_step);
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+ iterate_bvecq(&iter, iov_iter_count(&iter), NULL, NULL,
+ afs_dir_dump_step);
}
/*
@@ -203,9 +203,9 @@ static int afs_dir_check(struct afs_vnode *dvnode)
if (unlikely(!i_size))
return 0;
- iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
- checked = iterate_folioq(&iter, iov_iter_count(&iter), dvnode, NULL,
- afs_dir_check_step);
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+ checked = iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, NULL,
+ afs_dir_check_step);
if (checked != i_size) {
afs_dir_dump(dvnode);
return -EIO;
@@ -250,15 +250,15 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
if (dvnode->directory_size < i_size) {
size_t cur_size = dvnode->directory_size;
- ret = netfs_alloc_folioq_buffer(NULL,
- &dvnode->directory, &cur_size, i_size,
- mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+ ret = bvecq_expand_buffer(&dvnode->directory, &cur_size,
+ round_up(i_size, PAGE_SIZE),
+ mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
dvnode->directory_size = cur_size;
if (ret < 0)
return ret;
}
- iov_iter_folio_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
+ iov_iter_bvec_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
/* AFS requires us to perform the read of a directory synchronously as
* a single unit to avoid issues with the directory contents being
@@ -294,8 +294,8 @@ static ssize_t afs_read_single(struct afs_vnode *dvnode, struct file *file)
}
/*
- * Read the directory into a folio_queue buffer in one go, scrubbing the
- * previous contents. We return -ESTALE if the caller needs to call us again.
+ * Read the directory into the buffer in one go, scrubbing the previous
+ * contents. We return -ESTALE if the caller needs to call us again.
*/
ssize_t afs_read_dir(struct afs_vnode *dvnode, struct file *file)
__acquires(&dvnode->validate_lock)
@@ -483,7 +483,7 @@ static size_t afs_dir_iterate_step(void *iter_base, size_t progress, size_t len,
}
/*
- * Iterate through the directory folios.
+ * Iterate through the directory content.
*/
static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_ctx)
{
@@ -498,11 +498,11 @@ static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_c
if (i_size <= 0 || dir_ctx->pos >= i_size)
return 0;
- iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
iov_iter_advance(&iter, round_down(dir_ctx->pos, AFS_DIR_BLOCK_SIZE));
- iterate_folioq(&iter, iov_iter_count(&iter), dvnode, &ctx,
- afs_dir_iterate_step);
+ iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, &ctx,
+ afs_dir_iterate_step);
if (ctx.error == -ESTALE)
afs_invalidate_dir(dvnode, afs_dir_invalid_iter_stale);
@@ -2228,8 +2228,8 @@ static int afs_dir_writepages(struct address_space *mapping,
}
if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
- iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
- i_size_read(&dvnode->netfs.inode));
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
+ i_size_read(&dvnode->netfs.inode));
ret = netfs_writeback_single(mapping, wbc, &iter);
if (ret == 1)
ret = 0; /* Skipped write due to lock conflict. */
diff --git a/fs/afs/dir_edit.c b/fs/afs/dir_edit.c
index c31303059444a..de53d7ff82c38 100644
--- a/fs/afs/dir_edit.c
+++ b/fs/afs/dir_edit.c
@@ -10,7 +10,6 @@
#include <linux/namei.h>
#include <linux/pagemap.h>
#include <linux/iversion.h>
-#include <linux/folio_queue.h>
#include "internal.h"
#include "xdr_fs.h"
@@ -110,9 +109,8 @@ static void afs_clear_contig_bits(union afs_xdr_dir_block *block,
*/
static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, size_t block)
{
- struct folio_queue *fq;
struct afs_vnode *dvnode = iter->dvnode;
- struct folio *folio;
+ struct bvecq *bq;
size_t blpos = block * AFS_DIR_BLOCK_SIZE;
size_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter->fpos;
int ret;
@@ -120,41 +118,38 @@ static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, siz
if (dvnode->directory_size < blend) {
size_t cur_size = dvnode->directory_size;
- ret = netfs_alloc_folioq_buffer(
- NULL, &dvnode->directory, &cur_size, blend,
- mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+ ret = bvecq_expand_buffer(&dvnode->directory, &cur_size, blend,
+ mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
dvnode->directory_size = cur_size;
if (ret < 0)
goto fail;
}
- fq = iter->fq;
- if (!fq)
- fq = dvnode->directory;
+ bq = iter->bq;
+ if (!bq)
+ bq = dvnode->directory;
- /* Search the folio queue for the folio containing the block... */
- for (; fq; fq = fq->next) {
- for (int s = iter->fq_slot; s < folioq_count(fq); s++) {
- size_t fsize = folioq_folio_size(fq, s);
+ /* Search the contents for the region containing the block... */
+ for (; bq; bq = bq->next) {
+ for (int s = iter->bq_slot; s < bq->nr_slots; s++) {
+ struct bio_vec *bv = &bq->bv[s];
+ size_t bsize = bv->bv_len;
- if (blend <= fpos + fsize) {
+ if (blend <= fpos + bsize) {
/* ... and then return the mapped block. */
- folio = folioq_folio(fq, s);
- if (WARN_ON_ONCE(folio_pos(folio) != fpos))
- goto fail;
- iter->fq = fq;
- iter->fq_slot = s;
+ iter->bq = bq;
+ iter->bq_slot = s;
iter->fpos = fpos;
- return kmap_local_folio(folio, blpos - fpos);
+ return bvec_kmap_partial(bv, blpos - fpos);
}
- fpos += fsize;
+ fpos += bsize;
}
- iter->fq_slot = 0;
+ iter->bq_slot = 0;
}
fail:
- iter->fq = NULL;
- iter->fq_slot = 0;
+ iter->bq = NULL;
+ iter->bq_slot = 0;
afs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);
return NULL;
}
diff --git a/fs/afs/dir_search.c b/fs/afs/dir_search.c
index 11ebdfffcb1d3..e1f91b3ac6e25 100644
--- a/fs/afs/dir_search.c
+++ b/fs/afs/dir_search.c
@@ -66,12 +66,11 @@ bool afs_dir_init_iter(struct afs_dir_iter *iter, const struct qstr *name)
*/
union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t block)
{
- struct folio_queue *fq = iter->fq;
struct afs_vnode *dvnode = iter->dvnode;
- struct folio *folio;
+ struct bvecq *bq = iter->bq;
size_t blpos = block * AFS_DIR_BLOCK_SIZE;
size_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter->fpos;
- int slot = iter->fq_slot;
+ int slot = iter->bq_slot;
_enter("%zx,%d", block, slot);
@@ -80,36 +79,34 @@ union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t bl
if (dvnode->directory_size < blend)
goto fail;
- if (!fq || blpos < fpos) {
- fq = dvnode->directory;
+ if (!bq || blpos < fpos) {
+ bq = dvnode->directory;
slot = 0;
fpos = 0;
}
/* Search the folio queue for the folio containing the block... */
- for (; fq; fq = fq->next) {
- for (; slot < folioq_count(fq); slot++) {
- size_t fsize = folioq_folio_size(fq, slot);
+ for (; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ struct bio_vec *bv = &bq->bv[slot];
+ size_t bsize = bv->bv_len;
- if (blend <= fpos + fsize) {
+ if (blend <= fpos + bsize) {
/* ... and then return the mapped block. */
- folio = folioq_folio(fq, slot);
- if (WARN_ON_ONCE(folio_pos(folio) != fpos))
- goto fail;
- iter->fq = fq;
- iter->fq_slot = slot;
+ iter->bq = bq;
+ iter->bq_slot = slot;
iter->fpos = fpos;
- iter->block = kmap_local_folio(folio, blpos - fpos);
+ iter->block = bvec_kmap_partial(bv, blpos - fpos);
return iter->block;
}
- fpos += fsize;
+ fpos += bsize;
}
slot = 0;
}
fail:
- iter->fq = NULL;
- iter->fq_slot = 0;
+ iter->bq = NULL;
+ iter->bq_slot = 0;
afs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);
return NULL;
}
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 14f39a9bea6cf..634fbf8eb2120 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -683,7 +683,7 @@ void afs_evict_inode(struct inode *inode)
flush_delayed_work(&vnode->lock_work);
netfs_wait_for_outstanding_io(inode);
truncate_inode_pages_final(&inode->i_data);
- netfs_free_folioq_buffer(vnode->directory);
+ bvecq_put(vnode->directory);
if (vnode->symlink)
afs_evict_symlink(vnode);
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index d58273c00fbc7..51078cc9b0e55 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -710,7 +710,7 @@ struct afs_vnode {
#define AFS_VNODE_MODIFYING 10 /* Set if we're performing a modification op */
#define AFS_VNODE_DIR_READ 11 /* Set if we've read a dir's contents */
- struct folio_queue *directory; /* Directory contents */
+ struct bvecq *directory; /* Directory contents */
struct afs_symlink __rcu *symlink; /* Symlink content */
struct list_head wb_keys; /* List of keys available for writeback */
struct list_head pending_locks; /* locks waiting to be granted */
@@ -991,9 +991,9 @@ static inline void afs_invalidate_cache(struct afs_vnode *vnode, unsigned int fl
struct afs_dir_iter {
struct afs_vnode *dvnode;
union afs_xdr_dir_block *block;
- struct folio_queue *fq;
+ struct bvecq *bq;
unsigned int fpos;
- int fq_slot;
+ int bq_slot;
unsigned int loop_check;
u8 nr_slots;
u8 bucket;
diff --git a/fs/afs/symlink.c b/fs/afs/symlink.c
index 6b8c122877ca2..9a611efe6b264 100644
--- a/fs/afs/symlink.c
+++ b/fs/afs/symlink.c
@@ -56,7 +56,6 @@ void afs_evict_symlink(struct afs_vnode *vnode)
void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)
{
struct afs_symlink *symlink = op->create.symlink;
- size_t dsize = 0;
size_t size = strlen(symlink->content) + 1;
char *p;
@@ -66,13 +65,18 @@ void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)
if (!fscache_cookie_enabled(netfs_i_cookie(&vnode->netfs)))
return;
- if (netfs_alloc_folioq_buffer(NULL, &vnode->directory, &dsize, size,
- mapping_gfp_mask(vnode->netfs.inode.i_mapping)) < 0)
+ vnode->directory =
+ bvecq_alloc_buffer(PAGE_SIZE,
+ mapping_gfp_mask(vnode->netfs.inode.i_mapping),
+ false);
+ if (!vnode->directory)
return;
- vnode->directory_size = dsize;
- p = kmap_local_folio(folioq_folio(vnode->directory, 0), 0);
+ vnode->directory_size = size;
+ p = bvec_kmap_partial(&vnode->directory->bv[0], 0);
memcpy(p, symlink->content, size);
+ if (size < PAGE_SIZE)
+ memset(p + size, 0, PAGE_SIZE - size);
kunmap_local(p);
netfs_single_mark_inode_dirty(&vnode->netfs.inode);
}
@@ -94,17 +98,13 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
}
if (!vnode->directory) {
- size_t cur_size = 0;
-
- ret = netfs_alloc_folioq_buffer(NULL,
- &vnode->directory, &cur_size, PAGE_SIZE,
- mapping_gfp_mask(vnode->netfs.inode.i_mapping));
- vnode->directory_size = PAGE_SIZE - 1;
- if (ret < 0)
- return ret;
+ vnode->directory = bvecq_alloc_buffer(
+ PAGE_SIZE, mapping_gfp_mask(vnode->netfs.inode.i_mapping), false);
+ if (!vnode->directory)
+ return -ENOMEM;
}
- iov_iter_folio_queue(&iter, ITER_DEST, vnode->directory, 0, 0, PAGE_SIZE);
+ iov_iter_bvec_queue(&iter, ITER_DEST, vnode->directory, 0, 0, PAGE_SIZE);
/* AFS requires us to perform the read of a symlink as a single unit to
* avoid issues with the content being changed between reads.
@@ -126,7 +126,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
refcount_set(&symlink->ref, 1);
symlink->content[i_size] = 0;
- const char *s = kmap_local_folio(folioq_folio(vnode->directory, 0), 0);
+ const char *s = bvec_kmap_partial(&vnode->directory->bv[0], 0);
memcpy(symlink->content, s, i_size);
kunmap_local(s);
@@ -135,7 +135,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
}
if (!fscache_cookie_enabled(netfs_i_cookie(&vnode->netfs))) {
- netfs_free_folioq_buffer(vnode->directory);
+ bvecq_put(vnode->directory);
vnode->directory = NULL;
vnode->directory_size = 0;
}
@@ -248,14 +248,14 @@ int afs_symlink_writepages(struct address_space *mapping,
if (vnode->directory &&
atomic64_read(&vnode->cb_expires_at) != AFS_NO_CB_PROMISE) {
- iov_iter_folio_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
- i_size_read(&vnode->netfs.inode));
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
+ i_size_read(&vnode->netfs.inode));
ret = netfs_writeback_single(mapping, wbc, &iter);
}
if (ret == 0) {
netfs_wb_begin(&vnode->netfs, false);
- netfs_free_folioq_buffer(vnode->directory);
+ bvecq_put(vnode->directory);
vnode->directory = NULL;
vnode->directory_size = 0;
netfs_wb_end(&vnode->netfs);
diff --git a/fs/netfs/Makefile b/fs/netfs/Makefile
index 54834cde7e568..b1ea4439c1bb4 100644
--- a/fs/netfs/Makefile
+++ b/fs/netfs/Makefile
@@ -3,6 +3,7 @@
netfs-y := \
buffered_read.o \
buffered_write.o \
+ bvecq.o \
direct_read.o \
direct_write.o \
iterator.o \
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index e30bde80276a4..aa1e4f8d46ab3 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -215,7 +215,7 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
* otherwise we set the deprecated PG_private_2.
*/
static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,
- struct folio_queue **fq,
+ struct bvecq **bq,
unsigned int *offset,
int *slot,
size_t len,
@@ -225,21 +225,21 @@ static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,
struct folio *folio;
size_t fsize, overlap;
- if (!*fq)
+ if (!*bq)
break;
- if (*slot >= folioq_count(*fq)) {
- *fq = (*fq)->next;
+ if (!bvecq_acquire_slot(*bq, *slot)) {
+ *bq = bvecq_next(*bq);
*slot = 0;
*offset = 0;
continue;
}
/* Determine how much the subreq overlaps the folio, if at all. */
- fsize = folioq_folio_size(*fq, *slot);
+ fsize = (*bq)->bv[*slot].bv_len;
overlap = min(len, fsize - *offset);
if (overlap > 0 && copy) {
- folio = folioq_folio(*fq, *slot);
+ folio = bvec_folio(&(*bq)->bv[*slot]);
if (netfs_using_pgpriv2(rreq)) {
if (!folio_test_private_2(folio))
folio_start_private_2(folio);
@@ -275,7 +275,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
.cached_to[1] = ULLONG_MAX,
};
struct fscache_occupancy *occ = &_occ;
- struct folio_queue *fq = rreq->buffer.tail;
+ struct bvecq *bq = rreq->buffer.tail;
unsigned int offset = 0;
ssize_t size = rreq->len;
uoff_t start = rreq->start;
@@ -408,10 +408,10 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
if (size <= 0)
netfs_all_subreqs_queued(rreq);
- if (fq) {
+ if (bq) {
/* See if the cache indicated this should be cached. */
copy = test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
- netfs_mark_copy_to_cache(rreq, &fq, &slot, &offset, slice, copy);
+ netfs_mark_copy_to_cache(rreq, &bq, &slot, &offset, slice, copy);
}
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
@@ -479,8 +479,7 @@ void netfs_readahead(struct readahead_control *ractl)
* acquires a ref on each folio that we will need to release later -
* but we don't want to do that until after we've started the I/O.
*/
- added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl,
- rreq->debug_id, rreq->gfp);
+ added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl, rreq->gfp);
if (added < 0) {
ret = added;
goto cleanup_free;
@@ -503,15 +502,14 @@ EXPORT_SYMBOL(netfs_readahead);
/*
* Create a rolling buffer with a single occupying folio.
*/
-static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio,
- unsigned int rollbuf_flags)
+static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio)
{
ssize_t added;
- if (rolling_buffer_init(&rreq->buffer, rreq->debug_id, ITER_DEST, rreq->gfp) < 0)
+ if (rolling_buffer_init(&rreq->buffer, ITER_DEST, rreq->gfp, false) < 0)
return -ENOMEM;
- added = rolling_buffer_append(&rreq->buffer, folio, rollbuf_flags, rreq->gfp);
+ added = rolling_buffer_append(&rreq->buffer, folio, rreq->gfp);
if (added < 0)
return added;
rreq->submitted = rreq->start + added;
@@ -661,7 +659,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_readpage);
/* Set up the output buffer */
- ret = netfs_create_singular_buffer(rreq, folio, 0);
+ ret = netfs_create_singular_buffer(rreq, folio);
if (ret < 0)
goto discard;
@@ -818,7 +816,7 @@ int netfs_write_begin(struct netfs_inode *ctx,
trace_netfs_read(rreq, pos, len, netfs_read_trace_write_begin);
/* Set up the output buffer */
- ret = netfs_create_singular_buffer(rreq, folio, 0);
+ ret = netfs_create_singular_buffer(rreq, folio);
if (ret < 0)
goto error_put;
@@ -883,7 +881,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
trace_netfs_read(rreq, start, flen, netfs_read_trace_prefetch_for_write);
/* Set up the output buffer */
- ret = netfs_create_singular_buffer(rreq, folio, NETFS_ROLLBUF_PAGECACHE_MARK);
+ ret = netfs_create_singular_buffer(rreq, folio);
if (ret < 0)
goto error_put;
diff --git a/fs/netfs/bvecq.c b/fs/netfs/bvecq.c
new file mode 100644
index 0000000000000..e78882b5672b7
--- /dev/null
+++ b/fs/netfs/bvecq.c
@@ -0,0 +1,346 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Buffering helpers for bvec queues
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#include <linux/bvecq.h>
+#include "internal.h"
+
+void bvecq_dump(const struct bvecq *bq)
+{
+ int b = 0;
+
+ for (; bq; bq = bvecq_next(bq), b++) {
+ int skipz = 0;
+
+ pr_notice("BQ[%u] %u/%u\n", b, bq->nr_slots, bq->max_slots);
+ for (int s = 0; s < bq->nr_slots; s++) {
+ const struct bio_vec *bv = &bq->bv[s];
+
+ if (!bv->bv_page && !bv->bv_len && skipz < 2) {
+ skipz = 1;
+ continue;
+ }
+ if (skipz == 1)
+ pr_notice("BQ[%u:00-%02u] ...\n", b, s - 1);
+ skipz = 2;
+ pr_notice("BQ[%u:%02u] %10lx %04x %04x %u\n",
+ b, s,
+ bv->bv_page ? page_to_pfn(bv->bv_page) : 0,
+ bv->bv_offset, bv->bv_len,
+ bv->bv_page ? page_count(bv->bv_page) : 0);
+ }
+ }
+}
+EXPORT_SYMBOL(bvecq_dump);
+
+/**
+ * bvecq_alloc_one - Allocate a single bvecq node with unpopulated slots
+ * @nr_slots: Number of slots to allocate
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a single bvecq node and initialise the header. The allocation is
+ * rounded up to the size of the smallest slab granule that will accommodate it
+ * and all the remaining space is set as an inline slot array with max_slots
+ * set to the number of slots. The slot array is not initialised.
+ *
+ * If @for_writeback is set, then the emergency mempool may be used for
+ * allocation. If it does allocate from that pool, the maximum number of slots
+ * available will be BVECQ_POOL_SLOTS which may be less than requested. Also,
+ * if @for_writeback is set, the function will not fail - though it may have to
+ * wait.
+ *
+ * Return: The node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback)
+{
+ struct bvecq *bq;
+ const size_t pool_size = struct_size_t(struct bvecq, __bv, BVECQ_POOL_SLOTS);
+ size_t size;
+ bool from_pool = false;
+
+ size = kmalloc_size_roundup(struct_size(bq, __bv, nr_slots));
+ gfp &= ~(GFP_ZONEMASK | __GFP_THISNODE);
+
+ if (for_writeback) {
+ if (size != pool_size) {
+ gfp_t gfp_temp = gfp;
+
+ gfp_temp |= __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN;
+ gfp_temp &= ~(__GFP_DIRECT_RECLAIM | __GFP_IO);
+ bq = kmalloc(size, gfp_temp);
+ if (bq)
+ goto success;
+ }
+
+ bq = mempool_alloc(&netfs_bvecq_pool, gfp);
+ if (!bq)
+ return bq;
+ from_pool = true;
+ size = pool_size;
+ } else {
+ bq = kmalloc(size, gfp);
+ if (!bq)
+ return bq;
+ }
+
+success:
+ *bq = (struct bvecq) {
+ .ref = REFCOUNT_INIT(1),
+ .bv = bq->__bv,
+ .inline_bv = true,
+ .max_slots = (size - sizeof(*bq)) / sizeof(bq->__bv[0]),
+ .from_pool = from_pool,
+ };
+ netfs_stat(&netfs_n_bvecq);
+ return bq;
+}
+EXPORT_SYMBOL(bvecq_alloc_one);
+
+/**
+ * bvecq_alloc_chain - Allocate an unpopulated bvecq chain
+ * @nr_slots: Number of slots to allocate
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a chain of bvecq nodes providing at least the requested cumulative
+ * number of slots. Each node is a maximum of 4KiB in size.
+ *
+ * Return: The first node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp, bool for_writeback)
+{
+ struct bvecq *head = NULL, *tail = NULL;
+
+ _enter("%zu", nr_slots);
+
+ for (;;) {
+ struct bvecq *bq;
+
+ bq = bvecq_alloc_one(min(nr_slots, BVECQ_4KB_SLOTS), gfp, for_writeback);
+ if (!bq)
+ goto oom;
+
+ if (tail)
+ bvecq_append(tail, bq);
+ else
+ head = bq;
+ tail = bq;
+ if (tail->max_slots >= nr_slots)
+ break;
+ nr_slots -= tail->max_slots;
+ }
+
+ return head;
+oom:
+ bvecq_put(head);
+ return NULL;
+}
+EXPORT_SYMBOL(bvecq_alloc_chain);
+
+/**
+ * bvecq_alloc_buffer2 - Allocate a bvecq chain and populate with buffers
+ * @size: Target size of the buffer (can be 0 for an empty buffer)
+ * @pre_slots: Number of preamble slots to set aside
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a chain of bvecq nodes and populate the slots with sufficient pages
+ * to provide at least the requested amount of space, leaving the first
+ * @pre_slots slots unset. The pre-slots must all fit into the the first
+ * bvecq.
+ *
+ * The pages allocated may be compound pages larger than PAGE_SIZE and thus
+ * occupy fewer slots. The pages have their refcounts set to 1 and can be
+ * passed to MSG_SPLICE_PAGES.
+ *
+ * Return: The first node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp,
+ bool for_writeback)
+{
+ struct bvecq *head = NULL, *p = NULL;
+ size_t nr_per_bq = BVECQ_POOL_SLOTS;
+ size_t count = pre_slots + DIV_ROUND_UP(size, PAGE_SIZE);
+
+ _enter("%zx,%zx,%u", size, count, pre_slots);
+
+ if (WARN_ON_ONCE(pre_slots > nr_per_bq))
+ return NULL;
+
+ head = bvecq_alloc_chain(count, gfp, for_writeback);
+ if (!head)
+ return NULL;
+
+ p = head;
+ do {
+ struct page **pages;
+ size_t unused, want, got, slot;
+
+ if (!count)
+ break;
+ if (WARN_ON_ONCE(!p))
+ goto oom;
+
+ if (p->nr_slots == 0) {
+ /* Need to clear pre slots and pages[], so just clear all. */
+ memset(p->bv, 0, p->max_slots * sizeof(p->bv[0]));
+ p->mem_type = BVECQ_MEM_ALLOCED;
+ p->nr_slots = pre_slots;
+ count -= pre_slots;
+ pre_slots = 0;
+ if (!count)
+ break;
+ }
+
+ if (p->nr_slots >= p->max_slots) {
+ p = p->next;
+ continue;
+ }
+ unused = p->max_slots - p->nr_slots;
+
+ pages = (struct page **)&p->bv[p->max_slots];
+ pages -= unused;
+
+ want = min(count, unused);
+ got = alloc_pages_bulk(gfp, want, pages);
+ if (!got)
+ goto oom;
+
+ slot = p->nr_slots;
+ for (int i = 0; i < got; i++) {
+ set_page_count(pages[i], 1);
+ bvec_set_page(&p->bv[slot++], pages[i], PAGE_SIZE, 0);
+ }
+
+ bvecq_filled_to(p, slot);
+ count -= got;
+ } while (count > 0);
+
+ return head;
+oom:
+ bvecq_put(head);
+ return NULL;
+}
+EXPORT_SYMBOL(bvecq_alloc_buffer2);
+
+/*
+ * Free the page pointed to by a slot as necessary.
+ */
+static void bvecq_free_slot(struct bvecq *bq, unsigned int slot)
+{
+ struct page *page = bq->bv[slot].bv_page;
+
+ if (!page)
+ return;
+
+ switch (bq->mem_type) {
+ case BVECQ_MEM_EXTERNAL:
+ break;
+ case BVECQ_MEM_PAGECACHE:
+ put_page(page);
+ break;
+ case BVECQ_MEM_GUP:
+ unpin_user_page(page);
+ break;
+ case BVECQ_MEM_ALLOCED:
+ __free_pages(page, compound_order(page));
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ break;
+ }
+}
+
+/**
+ * bvecq_put - Put a ref on a bvec queue
+ * @bq: The start of the folio queue to free
+ *
+ * Put the ref(s) on the nodes in a bvec queue, freeing up the node and the
+ * page fragments it points to as the refcounts become zero.
+ */
+void bvecq_put(struct bvecq *bq)
+{
+ struct bvecq *next;
+
+ for (; bq; bq = next) {
+ if (!refcount_dec_and_test(&bq->ref))
+ break;
+ for (int slot = 0; slot < bq->nr_slots; slot++)
+ bvecq_free_slot(bq, slot);
+ next = bq->next;
+ netfs_stat_d(&netfs_n_bvecq);
+ if (bq->from_pool)
+ mempool_free(bq, &netfs_bvecq_pool);
+ else
+ kfree(bq);
+ }
+}
+EXPORT_SYMBOL(bvecq_put);
+
+/**
+ * bvecq_expand_buffer - Allocate buffer space into a bvec queue
+ * @_buffer: Pointer to the bvecq chain to expand (may point to a NULL; updated).
+ * @_cur_size: Current size of the buffer (updated).
+ * @size: Target size of the buffer.
+ * @gfp: The allocation constraints.
+ *
+ * Append extra pages to a buffer to increase its capacity to the @size
+ * specified. If the current tail has space, but is not of the
+ * BVECQ_MEM_ALLOCED memory type, a separate bvecq will be allocated to hold
+ * the new memory.
+ */
+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp)
+{
+ struct bvecq *tail = *_buffer;
+
+ size = round_up(size, PAGE_SIZE);
+ if (tail)
+ while (tail->next)
+ tail = tail->next;
+
+ while (*_cur_size < size) {
+ struct page *page;
+ size_t need = size - *_cur_size;
+ gfp_t gfp_add;
+ int order = 0;
+
+ if (!tail || bvecq_is_full(tail) || tail->mem_type != BVECQ_MEM_ALLOCED) {
+ struct bvecq *p;
+
+ p = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, false);
+ if (!p)
+ return -ENOMEM;
+ if (tail)
+ bvecq_append(tail, p);
+ else
+ *_buffer = p;
+ tail = p;
+ p->mem_type = BVECQ_MEM_ALLOCED;
+ }
+
+ if (need > PAGE_SIZE)
+ order = umin(ilog2(need) - PAGE_SHIFT, MAX_PAGECACHE_ORDER);
+
+ gfp_add = 0;
+ if (order > 0)
+ gfp_add |= __GFP_NORETRY | __GFP_NOWARN;
+ page = alloc_pages(gfp | __GFP_COMP | gfp_add, order);
+ if (!page && order > 0) {
+ page = alloc_pages(gfp | __GFP_COMP, 0);
+ order = 0;
+ }
+ if (!page)
+ return -ENOMEM;
+
+ bvec_set_page(&tail->bv[tail->nr_slots], page, PAGE_SIZE << order, 0);
+ *_cur_size += PAGE_SIZE << order;
+ bvecq_filled_to(tail, tail->nr_slots + 1);
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(bvecq_expand_buffer);
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index b8591abc90a98..583ffa967d5a9 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -7,7 +7,6 @@
#include <linux/slab.h>
#include <linux/seq_file.h>
-#include <linux/folio_queue.h>
#include <linux/netfs.h>
#include <linux/fscache.h>
#include <linux/fscache-cache.h>
@@ -45,7 +44,7 @@ extern struct list_head netfs_io_requests;
extern spinlock_t netfs_proc_lock;
extern mempool_t netfs_request_pool;
extern mempool_t netfs_subrequest_pool;
-extern mempool_t netfs_folioq_pool;
+extern mempool_t netfs_bvecq_pool;
#ifdef CONFIG_PROC_FS
static inline void netfs_proc_add_rreq(struct netfs_io_request *rreq)
@@ -70,8 +69,6 @@ static inline void netfs_proc_del_rreq(struct netfs_io_request *rreq) {}
/*
* misc.c
*/
-struct folio_queue *netfs_buffer_make_space(struct netfs_io_request *rreq,
- enum netfs_folioq_trace trace);
void netfs_reset_iter(struct netfs_io_subrequest *subreq);
void netfs_wake_collector(struct netfs_io_request *rreq);
void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq);
@@ -202,7 +199,7 @@ extern atomic_t netfs_n_wh_retry_write_req;
extern atomic_t netfs_n_wh_retry_write_subreq;
extern atomic_t netfs_n_wb_lock_skip;
extern atomic_t netfs_n_wb_lock_wait;
-extern atomic_t netfs_n_folioq;
+extern atomic_t netfs_n_bvecq;
int netfs_stats_show(struct seq_file *m, void *v);
diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index eb1efb17f53aa..31748526d5682 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -245,33 +245,34 @@ static size_t netfs_limit_xarray(const struct iov_iter *iter, size_t start_offse
}
/*
- * Select the span of a folio queue iterator we're going to use. Limit it by
- * both maximum size and maximum number of segments. Returns the size of the
- * span in bytes.
+ * Select the span of a bvecq iterator we're going to use. Limit it by both
+ * maximum size and maximum number of segments. Returns the size of the span
+ * in bytes.
*/
-static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offset,
- size_t max_size, size_t max_segs)
+static size_t netfs_limit_bvecq(const struct iov_iter *iter, size_t start_offset,
+ size_t max_size, size_t max_segs)
{
- const struct folio_queue *folioq = iter->folioq;
+ const struct bvecq *bq = iter->bvecq;
unsigned int nsegs = 0;
- unsigned int slot = iter->folioq_slot;
+ unsigned int slot = iter->bvecq_slot;
size_t span = 0, n = iter->count;
- if (WARN_ON(!iov_iter_is_folioq(iter)) ||
+ if (WARN_ON(!iov_iter_is_bvecq(iter)) ||
WARN_ON(start_offset > n) ||
n == 0)
return 0;
max_size = umin(max_size, n - start_offset);
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = bvecq_next(bq);
slot = 0;
}
start_offset += iter->iov_offset;
do {
- size_t flen = folioq_folio_size(folioq, slot);
+ size_t flen;
+ flen = bq->bv[slot].bv_len;
if (start_offset < flen) {
span += flen - start_offset;
nsegs++;
@@ -283,11 +284,11 @@ static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offse
break;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = bvecq_next(bq);
slot = 0;
}
- } while (folioq);
+ } while (bq);
return umin(span, max_size);
}
@@ -295,8 +296,8 @@ static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offse
size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
size_t max_size, size_t max_segs)
{
- if (iov_iter_is_folioq(iter))
- return netfs_limit_folioq(iter, start_offset, max_size, max_segs);
+ if (iov_iter_is_bvecq(iter))
+ return netfs_limit_bvecq(iter, start_offset, max_size, max_segs);
if (iov_iter_is_bvec(iter))
return netfs_limit_bvec(iter, start_offset, max_size, max_segs);
if (iov_iter_is_xarray(iter))
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 609e22e8f76a0..36a1d98e593a0 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -28,7 +28,7 @@ static struct kmem_cache *netfs_request_slab;
static struct kmem_cache *netfs_subrequest_slab;
mempool_t netfs_request_pool;
mempool_t netfs_subrequest_pool;
-mempool_t netfs_folioq_pool;
+mempool_t netfs_bvecq_pool;
#ifdef CONFIG_PROC_FS
LIST_HEAD(netfs_io_requests);
@@ -108,8 +108,9 @@ static int __init netfs_init(void)
{
int ret = -ENOMEM;
- if (mempool_init_kmalloc_pool(&netfs_folioq_pool, 100, sizeof(struct folio_queue)) < 0)
- goto error_folioq_pool;
+ if (mempool_init_kmalloc_pool(&netfs_bvecq_pool, 100,
+ struct_size_t(struct bvecq, __bv, BVECQ_POOL_SLOTS)) < 0)
+ goto error_bvecq_pool;
netfs_request_slab = kmem_cache_create("netfs_request",
sizeof(struct netfs_io_request), 0,
@@ -163,8 +164,8 @@ static int __init netfs_init(void)
error_reqpool:
kmem_cache_destroy(netfs_request_slab);
error_req:
- mempool_exit(&netfs_folioq_pool);
-error_folioq_pool:
+ mempool_exit(&netfs_bvecq_pool);
+error_bvecq_pool:
return ret;
}
fs_initcall(netfs_init);
@@ -177,6 +178,6 @@ static void __exit netfs_exit(void)
kmem_cache_destroy(netfs_subrequest_slab);
mempool_exit(&netfs_request_pool);
kmem_cache_destroy(netfs_request_slab);
- mempool_exit(&netfs_folioq_pool);
+ mempool_exit(&netfs_bvecq_pool);
}
module_exit(netfs_exit);
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index a3cd76d584b8b..40e0ff6491329 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -8,102 +8,6 @@
#include <linux/swap.h>
#include "internal.h"
-/**
- * netfs_alloc_folioq_buffer - Allocate buffer space into a folio queue
- * @mapping: Address space to set on the folio (or NULL).
- * @_buffer: Pointer to the folio queue to add to (may point to a NULL; updated).
- * @_cur_size: Current size of the buffer (updated).
- * @size: Target size of the buffer.
- * @gfp: The allocation constraints.
- */
-int netfs_alloc_folioq_buffer(struct address_space *mapping,
- struct folio_queue **_buffer,
- size_t *_cur_size, ssize_t size, gfp_t gfp)
-{
- struct folio_queue *tail = *_buffer, *p;
-
- size = round_up(size, PAGE_SIZE);
- if (*_cur_size >= size)
- return 0;
-
- if (tail)
- while (tail->next)
- tail = tail->next;
-
- do {
- struct folio *folio;
- int order = 0, slot;
-
- if (!tail || folioq_full(tail)) {
- p = netfs_folioq_alloc(0, GFP_NOFS, netfs_trace_folioq_alloc_buffer);
- if (!p)
- return -ENOMEM;
- if (tail) {
- tail->next = p;
- p->prev = tail;
- } else {
- *_buffer = p;
- }
- tail = p;
- }
-
- if (size - *_cur_size > PAGE_SIZE)
- order = umin(ilog2(size - *_cur_size) - PAGE_SHIFT,
- MAX_PAGECACHE_ORDER);
-
- folio = folio_alloc(gfp, order);
- if (!folio && order > 0)
- folio = folio_alloc(gfp, 0);
- if (!folio)
- return -ENOMEM;
-
- folio->mapping = mapping;
- folio->index = *_cur_size / PAGE_SIZE;
- trace_netfs_folio(folio, netfs_folio_trace_alloc_buffer);
- slot = folioq_append_mark(tail, folio);
- *_cur_size += folioq_folio_size(tail, slot);
- } while (*_cur_size < size);
-
- return 0;
-}
-EXPORT_SYMBOL(netfs_alloc_folioq_buffer);
-
-/**
- * netfs_free_folioq_buffer - Free a folio queue.
- * @fq: The start of the folio queue to free
- *
- * Free up a chain of folio_queues and, if marked, the marked folios they point
- * to.
- */
-void netfs_free_folioq_buffer(struct folio_queue *fq)
-{
- struct folio_queue *next;
- struct folio_batch fbatch;
-
- folio_batch_init(&fbatch);
-
- for (; fq; fq = next) {
- for (int slot = 0; slot < folioq_count(fq); slot++) {
- struct folio *folio = folioq_folio(fq, slot);
-
- if (!folio ||
- !folioq_is_marked(fq, slot))
- continue;
-
- trace_netfs_folio(folio, netfs_folio_trace_put);
- if (folio_batch_add(&fbatch, folio))
- folio_batch_release(&fbatch);
- }
-
- netfs_stat_d(&netfs_n_folioq);
- next = fq->next;
- kfree(fq);
- }
-
- folio_batch_release(&fbatch);
-}
-EXPORT_SYMBOL(netfs_free_folioq_buffer);
-
/*
* Reset the subrequest iterator to refer just to the region remaining to be
* read. The iterator may or may not have been advanced by socket ops or
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 2625efd48a9b1..6bbaaea693543 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -63,11 +63,11 @@ void netfs_cancel_copy_to_cache(struct netfs_io_request *rreq, struct folio *fol
* dirty and let writeback handle it.
*/
static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
- struct folio_queue *folioq,
+ struct bvecq *bq,
int slot)
{
struct netfs_folio *finfo;
- struct folio *folio = folioq_folio(folioq, slot);
+ struct folio *folio = bvec_folio(&bq->bv[slot]);
if (unlikely(folio_pos(folio) < rreq->abandon_to)) {
trace_netfs_folio(folio, netfs_folio_trace_abandon);
@@ -98,7 +98,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
trace_netfs_folio(folio, netfs_folio_trace_read_done);
}
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
} else {
// TODO: Use of PG_private_2 is deprecated.
if (folio_test_private_2(folio))
@@ -114,7 +114,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
folio_unlock(folio);
}
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
}
/*
@@ -122,21 +122,21 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
*/
void netfs_read_set_unlock_at(struct netfs_io_request *rreq)
{
- struct folio_queue *folioq = rreq->buffer.tail;
+ const struct bvecq *bq = rreq->buffer.tail;
unsigned int slot = rreq->buffer.first_tail_slot;
size_t cleaned_to = rreq->cleaned_to - rreq->start;
size_t progress_at = cleaned_to;
size_t minimum = 256 * 1024;
while (progress_at < rreq->len) {
- if (slot >= folioq_count(folioq)) {
- folioq = folioq->next;
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = bvecq_next(bq);
+ if (!bq)
break;
slot = 0;
}
- progress_at += folioq_folio_size(folioq, slot);
+ progress_at += bq->bv[slot].bv_len;
if (progress_at - cleaned_to >= minimum)
break;
slot++;
@@ -152,7 +152,7 @@ void netfs_read_set_unlock_at(struct netfs_io_request *rreq)
static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
unsigned int *notes)
{
- struct folio_queue *folioq = rreq->buffer.tail;
+ struct bvecq *bq = rreq->buffer.tail;
unsigned int slot = rreq->buffer.first_tail_slot;
uoff_t collected_to = rreq->collected_to;
@@ -161,9 +161,9 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
// TODO: Begin decryption
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&rreq->buffer);
- if (!folioq) {
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&rreq->buffer);
+ if (!bq) {
WRITE_ONCE(rreq->progress_at, rreq->len);
return;
}
@@ -182,13 +182,13 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
uoff_t fpos, fend;
size_t fsize;
- folio = folioq_folio(folioq, slot);
+ folio = bvec_folio(&bq->bv[slot]);
if (WARN_ONCE(!folio_test_locked(folio),
"R=%08x: folio %lx is not locked\n",
rreq->debug_id, folio->index))
trace_netfs_folio(folio, netfs_folio_trace_not_locked);
- fsize = folioq_folio_size(folioq, slot);
+ fsize = bq->bv[slot].bv_len;
fpos = folio_pos(folio);
fend = fpos + fsize;
@@ -198,29 +198,28 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
if (collected_to < fend)
break;
- netfs_unlock_read_folio(rreq, folioq, slot);
+ netfs_unlock_read_folio(rreq, bq, slot);
WRITE_ONCE(rreq->cleaned_to, fpos + fsize);
*notes |= MADE_PROGRESS;
- /* Clean up the head folioq. If we clear an entire folioq, then
- * we can get rid of it provided it's not also the tail folioq
+ /* Clean up the head bq. If we clear an entire bq, then
+ * we can get rid of it provided it's not also the tail bq
* being filled by the issuer.
*/
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&rreq->buffer);
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&rreq->buffer);
+ if (!bq)
goto done;
slot = 0;
- trace_netfs_folioq(folioq, netfs_trace_folioq_read_progress);
}
if (fpos + fsize >= collected_to)
break;
}
- rreq->buffer.tail = folioq;
+ rreq->buffer.tail = bq;
done:
rreq->buffer.first_tail_slot = slot;
diff --git a/fs/netfs/read_pgpriv2.c b/fs/netfs/read_pgpriv2.c
index 5280b606fda41..16d623ca9df3e 100644
--- a/fs/netfs/read_pgpriv2.c
+++ b/fs/netfs/read_pgpriv2.c
@@ -53,7 +53,7 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
trace_netfs_folio(folio, netfs_folio_trace_store_copy);
/* Attach the folio to the rolling buffer. */
- if (rolling_buffer_append(&creq->buffer, folio, 0, creq->gfp) < 0) {
+ if (rolling_buffer_append(&creq->buffer, folio, creq->gfp) < 0) {
set_bit(NETFS_RREQ_CANCEL_CACHING, &creq->flags);
folio_end_private_2(folio);
return;
@@ -173,13 +173,13 @@ void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)
*/
bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
{
- struct folio_queue *folioq = creq->buffer.tail;
+ struct bvecq *bq = creq->buffer.tail;
unsigned int slot = creq->buffer.first_tail_slot;
uoff_t collected_to = creq->collected_to;
bool made_progress = false;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&creq->buffer);
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&creq->buffer);
slot = 0;
}
@@ -188,7 +188,7 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
uoff_t fpos, fend;
size_t fsize, flen;
- folio = folioq_folio(folioq, slot);
+ folio = bvec_folio(&bq->bv[slot]);
if (WARN_ONCE(!folio_test_private_2(folio),
"R=%08x: folio %lx is not marked private_2\n",
creq->debug_id, folio->index))
@@ -211,15 +211,15 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
creq->cleaned_to = fpos + fsize;
made_progress = true;
- /* Clean up the head folioq. If we clear an entire folioq, then
- * we can get rid of it provided it's not also the tail folioq
+ /* Clean up the head bq. If we clear an entire bq, then
+ * we can get rid of it provided it's not also the tail bq
* being filled by the issuer.
*/
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&creq->buffer);
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&creq->buffer);
+ if (!bq)
goto done;
slot = 0;
}
@@ -228,7 +228,7 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
break;
}
- creq->buffer.tail = folioq;
+ creq->buffer.tail = bq;
done:
creq->buffer.first_tail_slot = slot;
return made_progress;
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index 5bd8dee5a8346..142c3fb8dab17 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -291,7 +291,7 @@ void netfs_retry_reads(struct netfs_io_request *rreq)
*/
void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
{
- struct folio_queue *p;
+ struct bvecq *p;
/* We have to wait for readahead refs to have been released before we
* can unlock any folios as the ref-dropper walks i_pages and the only
@@ -301,24 +301,25 @@ void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
netfs_wait_for_put_ra_refs(rreq);
for (p = rreq->buffer.tail; p; p = p->next) {
- for (int slot = 0; slot < folioq_count(p); slot++) {
- struct folio *folio = folioq_folio(p, slot);
+ for (int slot = rreq->buffer.first_tail_slot;
+ bvecq_acquire_slot(p, slot);
+ slot++) {
+ struct folio *folio;
- if (!folio)
+ if (!p->bv[slot].bv_page)
continue;
+
+ folio = bvec_folio(&p->bv[slot]);
netfs_cancel_copy_to_cache(rreq, folio);
- if (!folioq_is_marked2(p, slot)) {
- if (folio == rreq->no_unlock_folio &&
- test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO,
- &rreq->flags)) {
- _debug("no unlock");
- } else {
- trace_netfs_folio(folio,
- netfs_folio_trace_abandon);
- folio_unlock(folio);
- }
+ if (folio == rreq->no_unlock_folio &&
+ test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, &rreq->flags)) {
+ _debug("no unlock");
+ } else {
+ trace_netfs_folio(folio, netfs_folio_trace_abandon);
+ folio_unlock(folio);
}
}
+ rreq->buffer.first_tail_slot = 0;
}
}
diff --git a/fs/netfs/rolling_buffer.c b/fs/netfs/rolling_buffer.c
index d30d5ef6d86ec..66ce9add40122 100644
--- a/fs/netfs/rolling_buffer.c
+++ b/fs/netfs/rolling_buffer.c
@@ -12,96 +12,51 @@
#include <linux/slab.h>
#include "internal.h"
-static atomic_t debug_ids;
-
-/**
- * netfs_folioq_alloc - Allocate a folio_queue struct
- * @rreq_id: Associated debugging ID for tracing purposes
- * @gfp: Allocation constraints
- * @trace: Trace tag to indicate the purpose of the allocation
- *
- * Allocate, initialise and account the folio_queue struct and log a trace line
- * to mark the allocation.
- */
-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,
- unsigned int /*enum netfs_folioq_trace*/ trace)
-{
- struct folio_queue *fq;
-
- if (gfp == GFP_KERNEL)
- fq = mempool_alloc_noreserve(&netfs_folioq_pool, gfp);
- else
- fq = mempool_alloc(&netfs_folioq_pool, gfp);
- if (fq) {
- netfs_stat(&netfs_n_folioq);
- folioq_init(fq, rreq_id);
- fq->debug_id = atomic_inc_return(&debug_ids);
- trace_netfs_folioq(fq, trace);
- }
- return fq;
-}
-EXPORT_SYMBOL(netfs_folioq_alloc);
-
-/**
- * netfs_folioq_free - Free a folio_queue struct
- * @folioq: The object to free
- * @trace: Trace tag to indicate which free
- *
- * Free and unaccount the folio_queue struct.
- */
-void netfs_folioq_free(struct folio_queue *folioq,
- unsigned int /*enum netfs_trace_folioq*/ trace)
-{
- trace_netfs_folioq(folioq, trace);
- netfs_stat_d(&netfs_n_folioq);
- mempool_free(folioq, &netfs_folioq_pool);
-}
-EXPORT_SYMBOL(netfs_folioq_free);
-
/*
* Initialise a rolling buffer. We allocate an empty folio queue struct to so
* that the pointers can be independently driven by the producer and the
* consumer.
*/
-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,
- unsigned int direction, gfp_t gfp)
+int rolling_buffer_init(struct rolling_buffer *roll, unsigned int direction,
+ gfp_t gfp, bool for_writeback)
{
- struct folio_queue *fq;
+ struct bvecq *bq;
+
+ roll->for_writeback = for_writeback;
- fq = netfs_folioq_alloc(rreq_id, gfp, netfs_trace_folioq_rollbuf_init);
- if (!fq)
+ bq = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, for_writeback);
+ if (!bq)
return -ENOMEM;
- roll->head = fq;
- roll->tail = fq;
- iov_iter_folio_queue(&roll->iter, direction, fq, 0, 0, 0);
+ roll->head = bq;
+ roll->tail = bq;
+ iov_iter_bvec_queue(&roll->iter, direction, bq, 0, 0, 0);
return 0;
}
/*
- * Add another folio_queue to a rolling buffer if there's no space left.
+ * Add another bvecq to a rolling buffer if there's no space left.
*/
int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)
{
- struct folio_queue *fq, *head = roll->head;
+ struct bvecq *bq, *head = roll->head;
- if (!folioq_full(head))
+ if (!bvecq_is_full(head))
return 0;
- fq = netfs_folioq_alloc(head->rreq_id, gfp, netfs_trace_folioq_make_space);
- if (!fq)
+ bq = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, roll->for_writeback);
+ if (!bq)
return -ENOMEM;
- fq->prev = head;
- roll->head = fq;
- if (folioq_full(head)) {
+ roll->head = bq;
+ if (bvecq_is_full(head)) {
/* Make sure we don't leave the master iterator pointing to a
* block that might get immediately consumed.
*/
- if (roll->iter.folioq == head &&
- roll->iter.folioq_slot == folioq_nr_slots(head)) {
- roll->iter.folioq = fq;
- roll->iter.folioq_slot = 0;
+ if (roll->iter.bvecq == head &&
+ roll->iter.bvecq_slot == head->nr_slots) {
+ roll->iter.bvecq = bq;
+ roll->iter.bvecq_slot = 0;
}
}
@@ -110,7 +65,7 @@ int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)
* [!] NOTE: After we set head->next, the consumer is at liberty to
* immediately delete the old head.
*/
- smp_store_release(&head->next, fq);
+ bvecq_append(head, bq);
return 0;
}
@@ -119,55 +74,59 @@ int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)
*/
ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
struct readahead_control *ractl,
- unsigned int rreq_id, gfp_t gfp)
+ gfp_t gfp)
{
- struct folio_queue *fq;
- ssize_t loaded = 0;
+ struct bvecq *bq;
+ size_t loaded = 0;
while (ractl->_nr_pages - ractl->_batch_count > 0) {
+ struct page **pages;
unsigned int nr;
- /* Allocate a folioq to put some folios into and attach it to
+ /* Allocate a bvecq to put some folios into and attach it to
* the rolling buffer.
*/
- fq = netfs_folioq_alloc(rreq_id, gfp,
- netfs_trace_folioq_make_space);
- if (!fq)
+ bq = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, false);
+ if (!bq)
goto nomem_unlock;
- fq->prev = roll->head;
+ bq->mem_type = BVECQ_MEM_EXTERNAL; /* Folio cleanup handled separately. */
+
if (!roll->tail)
- roll->tail = fq;
+ roll->tail = bq;
else
- roll->head->next = fq;
- roll->head = fq;
+ bvecq_append(roll->head, bq);
+ roll->head = bq;
- /* Get a batch of folios and note their orders. */
- nr = __readahead_batch(ractl, (struct page **)fq->vec.folios,
- folioq_nr_slots(fq));
+ /* Get a bunch of folios and note their sizes. */
+ pages = (struct page **)(bq->bv + bq->max_slots);
+ pages -= bq->max_slots;
+ nr = __readahead_batch(ractl, pages, bq->max_slots);
if (WARN_ON_ONCE(!nr))
break;
- fq->vec.nr = nr;
for (int slot = 0; slot < nr; slot++) {
- struct folio *folio = folioq_folio(fq, slot);
- unsigned int order;
+ struct folio *folio = page_folio(pages[slot]);
+ size_t len = folio_size(folio);
- order = folio_order(folio);
- fq->orders[slot] = order;
- loaded += PAGE_SIZE << order;
+ bvec_set_folio(&bq->bv[slot], folio, len, 0);
+ loaded += len;
trace_netfs_folio(folio, netfs_folio_trace_read);
}
+
+ bvecq_filled_to(bq, nr);
}
WRITE_ONCE(roll->iter.count, loaded);
- iov_iter_folio_queue(&roll->iter, ITER_DEST, roll->tail, 0, 0, loaded);
+ iov_iter_bvec_queue(&roll->iter, ITER_DEST, roll->tail, 0, 0, loaded);
return loaded;
nomem_unlock:
- for (fq = roll->tail; fq; fq = fq->next) {
- for (int slot = 0; slot < folioq_count(fq); slot++) {
- folio_unlock(fq->vec.folios[slot]);
- folioq_mark(fq, slot);
+ for (bq = roll->tail; bq; bq = bq->next) {
+ for (int slot = 0; slot < bq->nr_slots; slot++) {
+ struct folio *folio = bvec_folio(&bq->bv[slot]);
+
+ folio_unlock(folio);
+ folio_put(folio);
}
}
rolling_buffer_clear(roll);
@@ -180,7 +139,7 @@ ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
* Append a folio to the rolling buffer.
*/
ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
- unsigned int flags, gfp_t gfp)
+ gfp_t gfp)
{
ssize_t size = folio_size(folio);
int slot;
@@ -188,16 +147,11 @@ ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
if (rolling_buffer_make_space(roll, gfp) < 0)
return -ENOMEM;
- slot = folioq_append(roll->head, folio);
- if (flags & ROLLBUF_MARK_1)
- folioq_mark(roll->head, slot);
- if (flags & ROLLBUF_MARK_2)
- folioq_mark2(roll->head, slot);
+ slot = roll->head->nr_slots;
+ bvec_set_folio(&roll->head->bv[slot], folio, size, 0);
+ bvecq_filled_to(roll->head, slot + 1);
WRITE_ONCE(roll->iter.count, roll->iter.count + size);
-
- /* Store the counter after setting the slot. */
- smp_store_release(&roll->next_head_slot, slot);
return size;
}
@@ -206,44 +160,23 @@ ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
* don't return the last buffer to keep the pointers independent, but return
* NULL instead.
*/
-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll)
+struct bvecq *rolling_buffer_delete_spent(struct rolling_buffer *roll)
{
- struct folio_queue *spent = roll->tail, *next = READ_ONCE(spent->next);
+ struct bvecq *spent = roll->tail, *next = bvecq_next(spent);
if (!next)
return NULL;
next->prev = NULL;
- netfs_folioq_free(spent, netfs_trace_folioq_delete);
roll->tail = next;
+ spent->next = NULL;
+ bvecq_put(spent);
return next;
}
/*
- * Clear out a rolling queue. Folios that have mark 1 set are put.
+ * Clear out a rolling queue.
*/
void rolling_buffer_clear(struct rolling_buffer *roll)
{
- struct folio_batch fbatch;
- struct folio_queue *p;
-
- folio_batch_init(&fbatch);
-
- while ((p = roll->tail)) {
- roll->tail = p->next;
- for (int slot = 0; slot < folioq_count(p); slot++) {
- struct folio *folio = folioq_folio(p, slot);
-
- if (!folio)
- continue;
- if (folioq_is_marked(p, slot)) {
- trace_netfs_folio(folio, netfs_folio_trace_put);
- if (!folio_batch_add(&fbatch, folio))
- folio_batch_release(&fbatch);
- }
- }
-
- netfs_folioq_free(p, netfs_trace_folioq_clear);
- }
-
- folio_batch_release(&fbatch);
+ bvecq_put(roll->tail);
}
diff --git a/fs/netfs/stats.c b/fs/netfs/stats.c
index 9a607c4e62dda..0ba6ce9295b25 100644
--- a/fs/netfs/stats.c
+++ b/fs/netfs/stats.c
@@ -46,7 +46,7 @@ atomic_t netfs_n_wh_retry_write_req;
atomic_t netfs_n_wh_retry_write_subreq;
atomic_t netfs_n_wb_lock_skip;
atomic_t netfs_n_wb_lock_wait;
-atomic_t netfs_n_folioq;
+atomic_t netfs_n_bvecq;
int netfs_stats_show(struct seq_file *m, void *v)
{
@@ -88,10 +88,10 @@ int netfs_stats_show(struct seq_file *m, void *v)
atomic_read(&netfs_n_rh_retry_read_subreq),
atomic_read(&netfs_n_wh_retry_write_req),
atomic_read(&netfs_n_wh_retry_write_subreq));
- seq_printf(m, "Objs : rr=%u sr=%u foq=%u wsc=%u\n",
+ seq_printf(m, "Objs : rr=%u sr=%u bq=%u wsc=%u\n",
atomic_read(&netfs_n_rh_rreq),
atomic_read(&netfs_n_rh_sreq),
- atomic_read(&netfs_n_folioq),
+ atomic_read(&netfs_n_bvecq),
atomic_read(&netfs_n_wh_wstream_conflict));
seq_printf(m, "WbLock : skip=%u wait=%u\n",
atomic_read(&netfs_n_wb_lock_skip),
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 6e8ea534230df..80e6a3194a609 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -114,11 +114,11 @@ int netfs_folio_written_back(struct folio *folio)
static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
unsigned int *notes)
{
- struct folio_queue *folioq = wreq->buffer.tail;
+ struct bvecq *bq = wreq->buffer.tail;
unsigned int slot = wreq->buffer.first_tail_slot;
uoff_t collected_to = wreq->collected_to;
- if (WARN_ON_ONCE(!folioq)) {
+ if (WARN_ON_ONCE(!bq)) {
pr_err("[!] Writeback unlock found empty rolling buffer!\n");
netfs_dump_request(wreq);
return;
@@ -130,9 +130,9 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
return;
}
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&wreq->buffer);
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&wreq->buffer);
+ if (!bq)
return;
slot = 0;
}
@@ -143,7 +143,7 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
uoff_t fpos, fend;
size_t fsize, flen;
- folio = folioq_folio(folioq, slot);
+ folio = bvec_folio(&bq->bv[slot]);
if (WARN_ONCE(!folio_test_writeback(folio),
"R=%08x: folio %lx is not under writeback\n",
wreq->debug_id, folio->index))
@@ -166,15 +166,15 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
wreq->cleaned_to = fpos + fsize;
*notes |= MADE_PROGRESS;
- /* Clean up the head folioq. If we clear an entire folioq, then
- * we can get rid of it provided it's not also the tail folioq
+ /* Clean up the head bq. If we clear an entire bq, then
+ * we can get rid of it provided it's not also the tail bq
* being filled by the issuer.
*/
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&wreq->buffer);
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&wreq->buffer);
+ if (!bq)
goto done;
slot = 0;
}
@@ -183,7 +183,7 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
break;
}
- wreq->buffer.tail = folioq;
+ wreq->buffer.tail = bq;
done:
wreq->buffer.first_tail_slot = slot;
}
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 3989b4ec0c4b4..9de73c0edad2f 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -107,7 +107,9 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
ictx = netfs_inode(wreq->inode);
if (is_cacheable)
fscache_begin_write_operation(&wreq->cache_resources, netfs_i_cookie(ictx));
- if (rolling_buffer_init(&wreq->buffer, wreq->debug_id, ITER_SOURCE, wreq->gfp) < 0)
+ if (rolling_buffer_init(&wreq->buffer, ITER_SOURCE, wreq->gfp,
+ (origin == NETFS_WRITEBACK ||
+ origin == NETFS_WRITEBACK_SINGLE)) < 0)
goto nomem;
wreq->cleaned_to = wreq->start;
@@ -162,12 +164,12 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
struct netfs_io_subrequest *subreq;
struct iov_iter *wreq_iter = &wreq->buffer.iter;
- /* Make sure we don't point the iterator at a used-up folio_queue
- * struct being used as a placeholder to prevent the queue from
- * collapsing. In such a case, extend the queue.
+ /* Make sure we don't point the iterator at a used-up bvecq struct
+ * being used as a placeholder to prevent the queue from collapsing.
+ * In such a case, extend the queue.
*/
- if (iov_iter_is_folioq(wreq_iter) &&
- wreq_iter->folioq_slot >= folioq_nr_slots(wreq_iter->folioq))
+ if (iov_iter_is_bvecq(wreq_iter) &&
+ !bvecq_acquire_slot(wreq_iter->bvecq, wreq_iter->bvecq_slot))
rolling_buffer_make_space(&wreq->buffer, wreq->gfp);
subreq = netfs_alloc_subrequest(wreq, stream->source);
@@ -450,7 +452,7 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
}
/* Attach the folio to the rolling buffer. */
- rolling_buffer_append(&wreq->buffer, folio, 0, wreq->gfp);
+ rolling_buffer_append(&wreq->buffer, folio, wreq->gfp);
/* Move the submission point forward to allow for write-streaming data
* not starting at the front of the page. We don't do write-streaming
@@ -624,129 +626,11 @@ int netfs_writepages(struct address_space *mapping,
}
EXPORT_SYMBOL(netfs_writepages);
-/*
- * Write some of a pending folio data back to the server and/or the cache.
- */
-static int netfs_write_folio_single(struct netfs_io_request *wreq,
- struct folio *folio)
-{
- struct netfs_io_stream *upload = &wreq->io_streams[0];
- struct netfs_io_stream *cache = &wreq->io_streams[1];
- struct netfs_io_stream *stream;
- size_t iter_off = 0;
- size_t fsize = folio_size(folio), flen;
- uoff_t fpos = folio_pos(folio);
- ssize_t ret;
- bool to_eof = false;
- bool no_debug = false;
-
- _enter("");
-
- flen = folio_size(folio);
- if (flen > wreq->i_size - fpos) {
- flen = wreq->i_size - fpos;
- folio_zero_segment(folio, flen, fsize);
- to_eof = true;
- } else if (flen == wreq->i_size - fpos) {
- to_eof = true;
- }
-
- _debug("folio %zx/%zx", flen, fsize);
-
- if (!upload->avail && !cache->avail) {
- trace_netfs_folio(folio, netfs_folio_trace_cancel_store);
- return 0;
- }
-
- if (!upload->construct)
- trace_netfs_folio(folio, netfs_folio_trace_store);
- else
- trace_netfs_folio(folio, netfs_folio_trace_store_plus);
-
- /* Attach the folio to the rolling buffer. */
- folio_get(folio);
- ret = rolling_buffer_append(&wreq->buffer, folio, NETFS_ROLLBUF_PUT_MARK, wreq->gfp);
- if (ret < 0) {
- folio_put(folio);
- return ret;
- }
-
- /* Move the submission point forward to allow for write-streaming data
- * not starting at the front of the page. We don't do write-streaming
- * with the cache as the cache requires DIO alignment.
- *
- * Also skip uploading for data that's been read and just needs copying
- * to the cache.
- */
- for (int s = 0; s < NR_IO_STREAMS; s++) {
- stream = &wreq->io_streams[s];
- stream->submit_off = 0;
- stream->submit_len = flen;
- if (!stream->avail) {
- stream->submit_off = UINT_MAX;
- stream->submit_len = 0;
- }
- }
-
- /* Attach the folio to one or more subrequests. For a big folio, we
- * could end up with thousands of subrequests if the wsize is small -
- * but we might need to wait during the creation of subrequests for
- * network resources (eg. SMB credits).
- */
- for (;;) {
- ssize_t part;
- size_t lowest_off = ULONG_MAX;
- int choose_s = -1;
-
- /* Always add to the lowest-submitted stream first. */
- for (int s = 0; s < NR_IO_STREAMS; s++) {
- stream = &wreq->io_streams[s];
- if (stream->submit_len > 0 &&
- stream->submit_off < lowest_off) {
- lowest_off = stream->submit_off;
- choose_s = s;
- }
- }
-
- if (choose_s < 0)
- break;
- stream = &wreq->io_streams[choose_s];
-
- /* Advance the iterator(s). */
- if (stream->submit_off > iter_off) {
- rolling_buffer_advance(&wreq->buffer, stream->submit_off - iter_off);
- iter_off = stream->submit_off;
- }
-
- atomic64_set(&wreq->issued_to, fpos + stream->submit_off);
- stream->submit_extendable_to = fsize - stream->submit_off;
- part = netfs_advance_write(wreq, stream, fpos + stream->submit_off,
- stream->submit_len, to_eof);
- stream->submit_off += part;
- if (part > stream->submit_len)
- stream->submit_len = 0;
- else
- stream->submit_len -= part;
- if (part > 0)
- no_debug = true;
- }
-
- wreq->buffer.iter.iov_offset = 0;
- if (fsize > iter_off)
- rolling_buffer_advance(&wreq->buffer, fsize - iter_off);
- atomic64_set(&wreq->issued_to, fpos + fsize);
-
- if (!no_debug)
- kdebug("R=%x: No submit", wreq->debug_id);
- _leave(" = 0");
- return 0;
-}
-
/**
* netfs_writeback_single - Write back a monolithic payload
* @mapping: The mapping to write from
* @wbc: Hints from the VM
- * @iter: Data to write, must be ITER_FOLIOQ.
+ * @iter: Data to write.
*
* Write a monolithic, non-pagecache object back to the server and/or
* the cache.
@@ -760,13 +644,8 @@ int netfs_writeback_single(struct address_space *mapping,
{
struct netfs_io_request *wreq;
struct netfs_inode *ictx = netfs_inode(mapping->host);
- struct folio_queue *fq;
- size_t size = iov_iter_count(iter);
int ret;
- if (WARN_ON_ONCE(!iov_iter_is_folioq(iter)))
- return -EIO;
-
if (!netfs_wb_begin(ictx, wbc->sync_mode == WB_SYNC_NONE)) {
/* The VFS will have undirtied the inode. */
netfs_single_mark_inode_dirty(&ictx->inode);
@@ -779,36 +658,42 @@ int netfs_writeback_single(struct address_space *mapping,
goto couldnt_start;
}
+ wreq->buffer.iter = *iter;
+ wreq->len = iov_iter_count(iter);
+ wreq->submitted = wreq->len;
+
__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
trace_netfs_write(wreq, netfs_write_trace_writeback_single);
netfs_stat(&netfs_n_wh_writepages);
- if (__test_and_set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
+ if (test_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
wreq->netfs_ops->begin_writeback(wreq);
- for (fq = (struct folio_queue *)iter->folioq; fq; fq = fq->next) {
- for (int slot = 0; slot < folioq_count(fq); slot++) {
- struct folio *folio = folioq_folio(fq, slot);
- size_t part = umin(folioq_folio_size(fq, slot), size);
+ for (int s = 0; s < NR_IO_STREAMS; s++) {
+ struct netfs_io_subrequest *subreq;
+ struct netfs_io_stream *stream = &wreq->io_streams[s];
- _debug("wbiter %lx %llx", folio->index, atomic64_read(&wreq->issued_to));
+ if (!stream->avail)
+ continue;
- ret = netfs_write_folio_single(wreq, folio);
- if (ret < 0)
- goto stop;
- size -= part;
- if (size <= 0)
- goto stop;
- }
+ netfs_prepare_write(wreq, stream, 0);
+
+ subreq = stream->construct;
+ subreq->len = wreq->len;
+ stream->submit_len = subreq->len;
+ stream->submit_extendable_to = round_up(wreq->len, PAGE_SIZE);
+
+ netfs_issue_write(wreq, stream);
}
-stop:
- for (int s = 0; s < NR_IO_STREAMS; s++)
- netfs_issue_write(wreq, &wreq->io_streams[s]);
netfs_all_subreqs_queued(wreq);
-
netfs_wake_collector(wreq);
+ /* TODO: Might want to be async here if WB_SYNC_NONE, but then need to
+ * wait before modifying.
+ */
+ ret = netfs_wait_for_write(wreq);
+
netfs_put_request(wreq, netfs_rreq_trace_put_return);
_leave(" = %d", ret);
return ret;
diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index 79e4e84f8985c..c8b1c23ef4942 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -289,7 +289,7 @@ struct smb_rqst {
struct kvec *rq_iov; /* array of kvecs */
unsigned int rq_nvec; /* number of kvecs in array */
struct iov_iter rq_iter; /* Data iterator */
- struct folio_queue *rq_buffer; /* Buffer for encryption */
+ struct bvecq *rq_buffer; /* Buffer for encryption */
};
struct mid_q_entry;
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index aa142420dae2e..317d10b229666 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -13,7 +13,7 @@
#include <linux/sort.h>
#include <crypto/aead.h>
#include <linux/fiemap.h>
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
#include <uapi/linux/magic.h>
#include "cifsfs.h"
#include "cifsglob.h"
@@ -4830,19 +4830,18 @@ crypt_message(struct TCP_Server_Info *server, int num_rqst,
}
/*
- * Copy data from an iterator to the folios in a folio queue buffer.
+ * Copy data from an iterator to the pages in a bvec queue buffer.
*/
-static bool cifs_copy_iter_to_folioq(struct iov_iter *iter, size_t size,
- struct folio_queue *buffer)
+static bool cifs_copy_iter_to_bvecq(struct iov_iter *iter, size_t size,
+ struct bvecq *buffer)
{
for (; buffer; buffer = buffer->next) {
- for (int s = 0; s < folioq_count(buffer); s++) {
- struct folio *folio = folioq_folio(buffer, s);
- size_t part = folioq_folio_size(buffer, s);
+ for (int s = 0; s < bvecq_nr_slots_acquire(buffer); s++) {
+ struct bio_vec *bv = &buffer->bv[s];
+ size_t part = umin(bv->bv_len, size);
- part = umin(part, size);
-
- if (copy_folio_from_iter(folio, 0, part, iter) != part)
+ if (copy_page_from_iter(bv->bv_page, bv->bv_offset,
+ part, iter) != part)
return false;
size -= part;
}
@@ -4854,7 +4853,7 @@ void
smb3_free_compound_rqst(int num_rqst, struct smb_rqst *rqst)
{
for (int i = 0; i < num_rqst; i++)
- netfs_free_folioq_buffer(rqst[i].rq_buffer);
+ bvecq_put(rqst[i].rq_buffer);
}
/*
@@ -4881,7 +4880,7 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,
for (int i = 1; i < num_rqst; i++) {
struct smb_rqst *old = &old_rq[i - 1];
struct smb_rqst *new = &new_rq[i];
- struct folio_queue *buffer = NULL;
+ struct bvecq *buffer = NULL;
size_t size = iov_iter_count(&old->rq_iter);
orig_len += smb_rqst_len(server, old);
@@ -4889,17 +4888,16 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,
new->rq_nvec = old->rq_nvec;
if (size > 0) {
- size_t cur_size = 0;
- rc = netfs_alloc_folioq_buffer(NULL, &buffer, &cur_size,
- size, GFP_NOFS);
- new->rq_buffer = buffer;
- if (rc < 0)
+ rc = -ENOMEM;
+ buffer = bvecq_alloc_buffer(size, GFP_NOFS, true);
+ if (!buffer)
goto err_free;
- iov_iter_folio_queue(&new->rq_iter, ITER_SOURCE,
- buffer, 0, 0, size);
+ new->rq_buffer = buffer;
+ iov_iter_bvec_queue(&new->rq_iter, ITER_SOURCE,
+ buffer, 0, 0, size);
- if (!cifs_copy_iter_to_folioq(&old->rq_iter, size, buffer)) {
+ if (!cifs_copy_iter_to_bvecq(&old->rq_iter, size, buffer)) {
rc = smb_EIO1(smb_eio_trace_tx_copy_iter_to_buf, size);
goto err_free;
}
@@ -4989,22 +4987,20 @@ decrypt_raw_data(struct TCP_Server_Info *server, char *buf,
}
static int
-cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,
- size_t skip, struct iov_iter *iter)
+cifs_copy_bvecq_to_iter(struct bvecq *bq, size_t data_size,
+ size_t skip, struct iov_iter *iter)
{
- for (; folioq; folioq = folioq->next) {
- for (int s = 0; s < folioq_count(folioq); s++) {
- struct folio *folio;
- size_t fsize, n, len;
+ for (; bq; bq = bq->next) {
+ for (int s = 0; s < bvecq_nr_slots_acquire(bq); s++) {
+ struct bio_vec *bv = &bq->bv[s];
+ size_t n, len;
if (data_size == 0)
return 0;
- folio = folioq_folio(folioq, s);
- fsize = folio_size(folio);
- len = umin(fsize - skip, data_size);
+ len = umin(bv->bv_len - skip, data_size);
- n = copy_folio_to_iter(folio, skip, len, iter);
+ n = copy_page_to_iter(bv->bv_page, bv->bv_offset + skip, len, iter);
if (n != len) {
cifs_dbg(VFS, "%s: something went wrong\n", __func__);
return smb_EIO2(smb_eio_trace_rx_copy_to_iter,
@@ -5026,7 +5022,7 @@ cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,
static int
handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
- char *buf, unsigned int buf_len, struct folio_queue *buffer,
+ char *buf, unsigned int buf_len, struct bvecq *buffer,
unsigned int buffer_len, bool is_offloaded)
{
unsigned int data_offset;
@@ -5136,8 +5132,8 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
}
/* Copy the data to the output I/O iterator. */
- rdata->result = cifs_copy_folioq_to_iter(buffer, data_len,
- cur_off, &rdata->subreq.io_iter);
+ rdata->result = cifs_copy_bvecq_to_iter(buffer, data_len,
+ cur_off, &rdata->subreq.io_iter);
if (rdata->result != 0) {
if (is_offloaded)
mid->mid_state = MID_RESPONSE_MALFORMED;
@@ -5176,7 +5172,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
struct smb2_decrypt_work {
struct work_struct decrypt;
struct TCP_Server_Info *server;
- struct folio_queue *buffer;
+ struct bvecq *buffer;
char *buf;
unsigned int len;
};
@@ -5190,7 +5186,7 @@ static void smb2_decrypt_offload(struct work_struct *work)
struct mid_q_entry *mid;
struct iov_iter iter;
- iov_iter_folio_queue(&iter, ITER_DEST, dw->buffer, 0, 0, dw->len);
+ iov_iter_bvec_queue(&iter, ITER_DEST, dw->buffer, 0, 0, dw->len);
rc = decrypt_raw_data(dw->server, dw->buf, dw->server->vals->read_rsp_size,
&iter, true);
if (rc) {
@@ -5239,7 +5235,7 @@ static void smb2_decrypt_offload(struct work_struct *work)
}
free_pages:
- netfs_free_folioq_buffer(dw->buffer);
+ bvecq_put(dw->buffer);
cifs_small_buf_release(dw->buf);
kfree(dw);
}
@@ -5285,12 +5281,12 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,
dw->len = len;
len = round_up(dw->len, PAGE_SIZE);
- size_t cur_size = 0;
- rc = netfs_alloc_folioq_buffer(NULL, &dw->buffer, &cur_size, len, GFP_NOFS);
- if (rc < 0)
+ rc = -ENOMEM;
+ dw->buffer = bvecq_alloc_buffer(len, GFP_NOFS, false);
+ if (!dw->buffer)
goto discard_data;
- iov_iter_folio_queue(&iter, ITER_DEST, dw->buffer, 0, 0, len);
+ iov_iter_bvec_queue(&iter, ITER_DEST, dw->buffer, 0, 0, len);
/* Read the data into the buffer and clear excess bufferage. */
rc = cifs_read_iter_from_socket(server, &iter, dw->len);
@@ -5348,7 +5344,7 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,
}
free_pages:
- netfs_free_folioq_buffer(dw->buffer);
+ bvecq_put(dw->buffer);
free_dw:
kfree(dw);
return rc;
diff --git a/fs/smb/smbdirect/connection.c b/fs/smb/smbdirect/connection.c
index afd31fa12a368..aa52ecdbef2ba 100644
--- a/fs/smb/smbdirect/connection.c
+++ b/fs/smb/smbdirect/connection.c
@@ -5,7 +5,7 @@
*/
#include "internal.h"
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
struct smbdirect_map_sges {
struct ib_sge *sge;
@@ -2015,6 +2015,69 @@ static ssize_t smbdirect_map_sges_from_bvec(struct iov_iter *iter,
return ret;
}
+/*
+ * Extract memory fragments from a BVECQ-class iterator and add them to an RDMA
+ * list. The fragments are not pinned.
+ */
+static ssize_t smbdirect_map_sges_from_bvecq(struct iov_iter *iter,
+ struct smbdirect_map_sges *state,
+ ssize_t maxsize)
+{
+ const struct bvecq *bq = iter->bvecq, *next;
+ unsigned int slot = iter->bvecq_slot;
+ ssize_t extracted = 0;
+ size_t offset = iter->iov_offset;
+
+ maxsize = umin(maxsize, iov_iter_count(iter));
+
+ do {
+ struct bio_vec *bv;
+ size_t bsize;
+
+ while (slot >= bq->nr_slots) {
+ next = bvecq_next(bq);
+ if (!next) {
+ if (WARN_ON_ONCE(maxsize > 0))
+ return -EIO;
+ goto out;
+ }
+ bq = next;
+ slot = 0;
+ }
+
+ bv = &bq->bv[slot];
+ bsize = bv->bv_len;
+
+ if (offset < bsize) {
+ size_t part = umin(maxsize, bsize - offset);
+ bool ok;
+
+ ok = smbdirect_map_sges_single_page(state,
+ bv->bv_page,
+ bv->bv_offset + offset,
+ part);
+ if (!ok)
+ return -EIO;
+
+ offset += part;
+ extracted += part;
+ maxsize -= part;
+ }
+
+ if (offset >= bsize) {
+ offset = 0;
+ slot++;
+ }
+ } while (state->num_sge < state->max_sge && maxsize > 0);
+
+out:
+ iter->bvecq = bq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= extracted;
+ return extracted;
+}
+
/*
* Extract fragments from a KVEC-class iterator and add them to an ib_sge list.
* This can deal with vmalloc'd buffers as well as kmalloc'd or static buffers.
@@ -2076,70 +2139,6 @@ static ssize_t smbdirect_map_sges_from_kvec(struct iov_iter *iter,
return ret;
}
-/*
- * Extract folio fragments from a FOLIOQ-class iterator and add them to an
- * ib_sge list. The folios are not pinned.
- */
-static ssize_t smbdirect_map_sges_from_folioq(struct iov_iter *iter,
- struct smbdirect_map_sges *state,
- ssize_t maxsize)
-{
- const struct folio_queue *folioq = iter->folioq;
- unsigned int slot = iter->folioq_slot;
- ssize_t ret = 0;
- size_t offset = iter->iov_offset;
-
- if (WARN_ON_ONCE(!folioq))
- return -EIO;
-
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- if (WARN_ON_ONCE(!folioq))
- return -EIO;
- slot = 0;
- }
-
- do {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t fsize = folioq_folio_size(folioq, slot);
-
- if (offset < fsize) {
- size_t part = umin(maxsize, fsize - offset);
- bool ok;
-
- ok = smbdirect_map_sges_single_page(state,
- folio_page(folio, 0),
- offset,
- part);
- if (!ok)
- return -EIO;
-
- offset += part;
- ret += part;
- maxsize -= part;
- }
-
- if (offset >= fsize) {
- offset = 0;
- slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- if (!folioq->next) {
- WARN_ON_ONCE(ret < iter->count);
- break;
- }
- folioq = folioq->next;
- slot = 0;
- }
- }
- } while (state->num_sge < state->max_sge && maxsize > 0);
-
- iter->folioq = folioq;
- iter->folioq_slot = slot;
- iter->iov_offset = offset;
- iter->count -= ret;
- return ret;
-}
-
/*
* Extract page fragments from up to the given amount of the source iterator
* and build up an ib_sge list that refers to all of those bits. The ib_sge list
@@ -2164,12 +2163,12 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,
case ITER_BVEC:
ret = smbdirect_map_sges_from_bvec(iter, state, len);
break;
+ case ITER_BVECQ:
+ ret = smbdirect_map_sges_from_bvecq(iter, state, len);
+ break;
case ITER_KVEC:
ret = smbdirect_map_sges_from_kvec(iter, state, len);
break;
- case ITER_FOLIOQ:
- ret = smbdirect_map_sges_from_folioq(iter, state, len);
- break;
default:
WARN_ONCE(1, "iov_iter_type[%u]\n", iov_iter_type(iter));
return -EIO;
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index fc566ee1c1ff1..df9ce860e7874 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -343,4 +343,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)
return page_to_phys(bvec->bv_page) + bvec->bv_offset;
}
+/**
+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ * @offset: Offset into bvec
+ *
+ * Map the page containing the byte at @offset into the kernel virtual address
+ * space. The caller is responsible for making sure this doesn't overrun.
+ *
+ * Call kunmap_local on the returned address to unmap.
+ */
+static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)
+{
+ offset += bvec->bv_offset;
+
+ return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
+ (offset & ~PAGE_MASK);
+}
+
#endif /* __LINUX_BVEC_H */
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
new file mode 100644
index 0000000000000..b984aaa449088
--- /dev/null
+++ b/include/linux/bvecq.h
@@ -0,0 +1,166 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Implementation of a segmented queue of bio_vec[].
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#ifndef _LINUX_BVECQ_H
+#define _LINUX_BVECQ_H
+
+#include <linux/bvec.h>
+
+/*
+ * The type of memory retention used by the elements in bvecq->bv[] and how to
+ * clean it up.
+ */
+enum bvecq_mem {
+ BVECQ_MEM_EXTERNAL, /* Externally retained memory - no freeing */
+ BVECQ_MEM_PAGECACHE, /* Ref'd pagecache pages - must put */
+ BVECQ_MEM_GUP, /* Pinned memory from get_user_pages() - unpin */
+ BVECQ_MEM_ALLOCED, /* Memory alloc'd by bvecq - can be freed/pooled */
+} __mode(byte);
+
+/*
+ * Segmented bio_vec queue.
+ *
+ * These can be linked together to form messages of indefinite length and
+ * iterated over with an ITER_BVECQ iterator. The list is non-circular; next
+ * and prev are NULL at the ends.
+ *
+ * The bv pointer points to the bio_vec array; this may be __bv if allocated
+ * together. The caller is responsible for determining whether or not this is
+ * the case as the array pointed to by bv may be follow on directly from the
+ * bvecq by accident of allocation (ie. ->bv == ->__bv is *not* sufficient to
+ * determine this).
+ */
+struct bvecq {
+ struct bvecq *next; /* Next bvec in the list or NULL */
+ struct bvecq *prev; /* Prev bvec in the list or NULL */
+ refcount_t ref;
+ u32 priv; /* Private data */
+ u16 nr_slots; /* Number of elements in bv[] used */
+ u16 max_slots; /* Number of elements allocated in bv[] */
+ enum bvecq_mem mem_type:3; /* What sort of memory and how to free it */
+ bool inline_bv:1; /* T if __bv[] is being used */
+ bool from_pool:1; /* T if bvecq from mempool */
+ struct bio_vec *bv; /* Pointer to array of page fragments */
+ struct bio_vec __bv[]; /* Default array (if ->inline_bv) */
+};
+
+/* Number of slots in a 512-byte mempool-backed bvecq. */
+#define BVECQ_POOL_SLOTS ((512 - sizeof(struct bvecq)) / sizeof(struct bio_vec))
+
+/* Number of slots in a 4K bvecq. */
+#define BVECQ_4KB_SLOTS ((4096 - sizeof(struct bvecq)) / sizeof(struct bio_vec))
+
+void bvecq_dump(const struct bvecq *bq);
+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback);
+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp, bool for_writeback);
+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp,
+ bool for_writeback);
+void bvecq_put(struct bvecq *bq);
+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp);
+
+/**
+ * bvecq_alloc_buffer - Allocate a bvecq chain and populate with buffers
+ * @size: Target size of the buffer (can be 0 for an empty buffer)
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Wrapper around %bvecq_alloc_buffer2().
+ */
+static inline struct bvecq *bvecq_alloc_buffer(size_t size, gfp_t gfp, bool for_writeback)
+{
+ return bvecq_alloc_buffer2(size, 0, gfp, for_writeback);
+}
+
+/**
+ * bvecq_get - Get a ref on a bvecq
+ * @bq: The bvecq to get a ref on
+ */
+static inline struct bvecq *bvecq_get(struct bvecq *bq)
+{
+ refcount_inc(&bq->ref);
+ return bq;
+}
+
+/**
+ * bvecq_is_full - Determine if a bvecq is full
+ * @bvecq: The object to query
+ *
+ * Return: true if full; false if not.
+ */
+static inline bool bvecq_is_full(const struct bvecq *bvecq)
+{
+ return bvecq->nr_slots >= bvecq->max_slots;
+}
+
+/**
+ * bvecq_filled_to - Release filled slots with release barrier
+ * @bvecq: The object modified
+ * @to: The latest slot filled + 1
+ */
+static inline void bvecq_filled_to(struct bvecq *bvecq, unsigned int to)
+{
+ /* Set the slot counter after filling the slot */
+ smp_store_release(&bvecq->nr_slots, to);
+}
+
+/**
+ * bvecq_nr_slots_acquire - Get the number of filled slots with acquire barrier
+ * @bvecq: The object to query
+ *
+ * Return: The number of filled slots
+ */
+static inline unsigned int bvecq_nr_slots_acquire(const struct bvecq *bvecq)
+{
+ /* Read the slot counter before looking at the slot */
+ return smp_load_acquire(&bvecq->nr_slots);
+}
+
+/**
+ * bvecq_acquire_slot - Determine if a slot is valid with acquire barrier
+ * @bvecq: The object to query
+ * @slot: The next slot
+ *
+ * Return: true if valid; false if might not be valid
+ */
+static inline bool bvecq_acquire_slot(const struct bvecq *bvecq, unsigned int slot)
+{
+ /* Read the slot counter before looking at the slot */
+ return slot < bvecq_nr_slots_acquire(bvecq);
+}
+
+/**
+ * bvecq_append - Get the next bvecq with appropriate barrier
+ * @to: The bvecq to append to
+ * @add: The bvecq to append
+ *
+ * Attach a new bvecq to a chain using an appropriate barrier to protect the
+ * write.
+ *
+ * [!] Note that this function transfers the caller's ref to the chain.
+ */
+static inline void bvecq_append(struct bvecq *to, struct bvecq *add)
+{
+ add->prev = to;
+
+ /* Make sure the initialisation is stored before the next pointer. */
+ smp_store_release(&to->next, add);
+}
+
+/**
+ * bvecq_next - Get the next bvecq with appropriate barrier
+ * @bq: The bvecq to start from
+ *
+ * Return the next bvecq in a chain, using an appropriate barrier to protect
+ * the access.
+ */
+static inline struct bvecq *bvecq_next(const struct bvecq *bq)
+{
+ /* Read the contents of the next node after the pointer to it. */
+ return smp_load_acquire(&bq->next);
+}
+
+#endif /* _LINUX_BVECQ_H */
diff --git a/include/linux/folio_queue.h b/include/linux/folio_queue.h
deleted file mode 100644
index f6d5f1f127c9f..0000000000000
--- a/include/linux/folio_queue.h
+++ /dev/null
@@ -1,282 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/* Queue of folios definitions
- *
- * Copyright (C) 2024 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells@redhat.com)
- *
- * See:
- *
- * Documentation/core-api/folio_queue.rst
- *
- * for a description of the API.
- */
-
-#ifndef _LINUX_FOLIO_QUEUE_H
-#define _LINUX_FOLIO_QUEUE_H
-
-#include <linux/folio_batch.h>
-#include <linux/mm.h>
-
-/*
- * Segment in a queue of running buffers. Each segment can hold a number of
- * folios and a portion of the queue can be referenced with the ITER_FOLIOQ
- * iterator. The possibility exists of inserting non-folio elements into the
- * queue (such as gaps).
- *
- * Explicit prev and next pointers are used instead of a list_head to make it
- * easier to add segments to tail and remove them from the head without the
- * need for a lock.
- */
-struct folio_queue {
- struct folio_batch vec; /* Folios in the queue segment */
- u8 orders[FOLIO_BATCH_SIZE]; /* Order of each folio */
- struct folio_queue *next; /* Next queue segment or NULL */
- struct folio_queue *prev; /* Previous queue segment of NULL */
- unsigned long marks; /* 1-bit mark per folio */
- unsigned long marks2; /* Second 1-bit mark per folio */
-#if FOLIO_BATCH_SIZE > BITS_PER_LONG
-#error marks is not big enough
-#endif
- unsigned int rreq_id;
- unsigned int debug_id;
-};
-
-/**
- * folioq_init - Initialise a folio queue segment
- * @folioq: The segment to initialise
- * @rreq_id: The request identifier to use in tracelines.
- *
- * Initialise a folio queue segment and set an identifier to be used in traces.
- *
- * Note that the folio pointers are left uninitialised.
- */
-static inline void folioq_init(struct folio_queue *folioq, unsigned int rreq_id)
-{
- folio_batch_init(&folioq->vec);
- folioq->next = NULL;
- folioq->prev = NULL;
- folioq->marks = 0;
- folioq->marks2 = 0;
- folioq->rreq_id = rreq_id;
- folioq->debug_id = 0;
-}
-
-/**
- * folioq_nr_slots: Query the capacity of a folio queue segment
- * @folioq: The segment to query
- *
- * Query the number of folios that a particular folio queue segment might hold.
- * [!] NOTE: This must not be assumed to be the same for every segment!
- */
-static inline unsigned int folioq_nr_slots(const struct folio_queue *folioq)
-{
- return FOLIO_BATCH_SIZE;
-}
-
-/**
- * folioq_count: Query the occupancy of a folio queue segment
- * @folioq: The segment to query
- *
- * Query the number of folios that have been added to a folio queue segment.
- * Note that this is not decreased as folios are removed from a segment.
- */
-static inline unsigned int folioq_count(struct folio_queue *folioq)
-{
- return folio_batch_count(&folioq->vec);
-}
-
-/**
- * folioq_full: Query if a folio queue segment is full
- * @folioq: The segment to query
- *
- * Query if a folio queue segment is fully occupied. Note that this does not
- * change if folios are removed from a segment.
- */
-static inline bool folioq_full(struct folio_queue *folioq)
-{
- //return !folio_batch_space(&folioq->vec);
- return folioq_count(folioq) >= folioq_nr_slots(folioq);
-}
-
-/**
- * folioq_is_marked: Check first folio mark in a folio queue segment
- * @folioq: The segment to query
- * @slot: The slot number of the folio to query
- *
- * Determine if the first mark is set for the folio in the specified slot in a
- * folio queue segment.
- */
-static inline bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot)
-{
- return test_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_mark: Set the first mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Set the first mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_mark(struct folio_queue *folioq, unsigned int slot)
-{
- set_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_unmark: Clear the first mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Clear the first mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_unmark(struct folio_queue *folioq, unsigned int slot)
-{
- clear_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_is_marked2: Check second folio mark in a folio queue segment
- * @folioq: The segment to query
- * @slot: The slot number of the folio to query
- *
- * Determine if the second mark is set for the folio in the specified slot in a
- * folio queue segment.
- */
-static inline bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot)
-{
- return test_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_mark2: Set the second mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Set the second mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_mark2(struct folio_queue *folioq, unsigned int slot)
-{
- set_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_unmark2: Clear the second mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Clear the second mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_unmark2(struct folio_queue *folioq, unsigned int slot)
-{
- clear_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_append: Add a folio to a folio queue segment
- * @folioq: The segment to add to
- * @folio: The folio to add
- *
- * Add a folio to the tail of the sequence in a folio queue segment, increasing
- * the occupancy count and returning the slot number for the folio just added.
- * The folio size is extracted and stored in the queue and the marks are left
- * unmodified.
- *
- * Note that it's left up to the caller to check that the segment capacity will
- * not be exceeded and to extend the queue.
- */
-static inline unsigned int folioq_append(struct folio_queue *folioq, struct folio *folio)
-{
- unsigned int slot = folioq->vec.nr++;
-
- folioq->vec.folios[slot] = folio;
- folioq->orders[slot] = folio_order(folio);
- return slot;
-}
-
-/**
- * folioq_append_mark: Add a folio to a folio queue segment
- * @folioq: The segment to add to
- * @folio: The folio to add
- *
- * Add a folio to the tail of the sequence in a folio queue segment, increasing
- * the occupancy count and returning the slot number for the folio just added.
- * The folio size is extracted and stored in the queue, the first mark is set
- * and and the second and third marks are left unmodified.
- *
- * Note that it's left up to the caller to check that the segment capacity will
- * not be exceeded and to extend the queue.
- */
-static inline unsigned int folioq_append_mark(struct folio_queue *folioq, struct folio *folio)
-{
- unsigned int slot = folioq->vec.nr++;
-
- folioq->vec.folios[slot] = folio;
- folioq->orders[slot] = folio_order(folio);
- folioq_mark(folioq, slot);
- return slot;
-}
-
-/**
- * folioq_folio: Get a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the folio in the specified slot from a folio queue segment. Note
- * that no bounds check is made and if the slot hasn't been added into yet, the
- * pointer will be undefined. If the slot has been cleared, NULL will be
- * returned.
- */
-static inline struct folio *folioq_folio(const struct folio_queue *folioq, unsigned int slot)
-{
- return folioq->vec.folios[slot];
-}
-
-/**
- * folioq_folio_order: Get the order of a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the order of the folio in the specified slot from a folio queue
- * segment. Note that no bounds check is made and if the slot hasn't been
- * added into yet, the order returned will be 0.
- */
-static inline unsigned int folioq_folio_order(const struct folio_queue *folioq, unsigned int slot)
-{
- return folioq->orders[slot];
-}
-
-/**
- * folioq_folio_size: Get the size of a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the size of the folio in the specified slot from a folio queue
- * segment. Note that no bounds check is made and if the slot hasn't been
- * added into yet, the size returned will be PAGE_SIZE.
- */
-static inline size_t folioq_folio_size(const struct folio_queue *folioq, unsigned int slot)
-{
- return PAGE_SIZE << folioq_folio_order(folioq, slot);
-}
-
-/**
- * folioq_clear: Clear a folio from a folio queue segment
- * @folioq: The segment to clear
- * @slot: The folio slot to clear
- *
- * Clear a folio from a sequence in a folio queue segment and clear its marks.
- * The occupancy count is left unchanged.
- */
-static inline void folioq_clear(struct folio_queue *folioq, unsigned int slot)
-{
- folioq->vec.folios[slot] = NULL;
- folioq_unmark(folioq, slot);
- folioq_unmark2(folioq, slot);
-}
-
-#endif /* _LINUX_FOLIO_QUEUE_H */
diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h
index f9a17fbbd3980..39ffd5878f505 100644
--- a/include/linux/iov_iter.h
+++ b/include/linux/iov_iter.h
@@ -10,7 +10,7 @@
#include <linux/uio.h>
#include <linux/bvec.h>
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
void *priv, void *priv2);
@@ -142,56 +142,65 @@ size_t iterate_bvec(struct iov_iter *iter, size_t len, void *priv, void *priv2,
}
/*
- * Handle ITER_FOLIOQ.
+ * Handle ITER_BVECQ.
*/
static __always_inline
-size_t iterate_folioq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
- iov_step_f step)
+size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
+ iov_step_f step)
{
- const struct folio_queue *folioq = iter->folioq;
- unsigned int slot = iter->folioq_slot;
+ const struct bvecq *bq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
size_t progress = 0, skip = iter->iov_offset;
- if (slot == folioq_nr_slots(folioq)) {
- /* The iterator may have been extended. */
- folioq = folioq->next;
- slot = 0;
- }
-
do {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t part, remain = 0, consumed;
- size_t fsize;
+ const struct bio_vec *bvec;
+ struct page *page;
+ size_t poff, plen;
void *base;
- if (!folio)
- break;
+ if (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
+ slot = 0;
+ continue;
+ }
+
+ bvec = &bq->bv[slot];
+ /*
+ * The caller must ensure that a slot with bv_len>0 has a valid
+ * bv_page.
+ */
+ page = bvec->bv_page + (bvec->bv_offset + skip) / PAGE_SIZE;
+ poff = (bvec->bv_offset + skip) % PAGE_SIZE;
+ plen = min(bvec->bv_len - skip, len);
+
+ while (plen > 0) {
+ size_t part, remain, consumed;
- fsize = folioq_folio_size(folioq, slot);
- if (skip < fsize) {
- base = kmap_local_folio(folio, skip);
- part = umin(len, PAGE_SIZE - skip % PAGE_SIZE);
+ part = min(plen, PAGE_SIZE - poff);
+ base = kmap_local_page(page) + poff;
remain = step(base, progress, part, priv, priv2);
kunmap_local(base);
+
consumed = part - remain;
- len -= consumed;
progress += consumed;
skip += consumed;
+ len -= consumed;
+ if (!len || remain)
+ goto stop;
+ page++;
+ poff = 0;
+ plen -= consumed;
}
- if (skip >= fsize) {
- skip = 0;
- slot++;
- if (slot == folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
- slot = 0;
- }
- }
- if (remain)
- break;
+
+ skip = 0;
+ slot++;
} while (len);
- iter->folioq_slot = slot;
- iter->folioq = folioq;
+stop:
+ iter->bvecq_slot = slot;
+ iter->bvecq = bq;
iter->iov_offset = skip;
iter->count -= progress;
return progress;
@@ -306,8 +315,8 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
- if (iov_iter_is_folioq(iter))
- return iterate_folioq(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
return iterate_xarray(iter, len, priv, priv2, step);
return iterate_discard(iter, len, priv, priv2, step);
@@ -342,7 +351,7 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
* buffer is presented in segments, which for kernel iteration are broken up by
* physical pages and mapped, with the mapped address being presented.
*
- * [!] Note This will only handle BVEC, KVEC, FOLIOQ, XARRAY and DISCARD-type
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, XARRAY and DISCARD-type
* iterators; it will not handle UBUF or IOVEC-type iterators.
*
* A step functions, @step, must be provided, one for handling mapped kernel
@@ -370,8 +379,8 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
- if (iov_iter_is_folioq(iter))
- return iterate_folioq(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
return iterate_xarray(iter, len, priv, priv2, step);
return iterate_discard(iter, len, priv, priv2, step);
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 67e010b6994b3..31a80f4377d75 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -17,13 +17,13 @@
#include <linux/workqueue.h>
#include <linux/fs.h>
#include <linux/pagemap.h>
+#include <linux/bvecq.h>
#include <linux/uio.h>
#include <linux/rolling_buffer.h>
enum netfs_sreq_ref_trace;
typedef struct mempool mempool_t;
struct fscache_occupancy;
-struct folio_queue;
/**
* folio_start_private_2 - Start an fscache write on a folio. [DEPRECATED]
@@ -469,18 +469,6 @@ void netfs_end_io_write(struct inode *inode);
int netfs_start_io_direct(struct inode *inode);
void netfs_end_io_direct(struct inode *inode);
-/* Miscellaneous APIs. */
-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,
- unsigned int trace /*enum netfs_folioq_trace*/);
-void netfs_folioq_free(struct folio_queue *folioq,
- unsigned int trace /*enum netfs_trace_folioq*/);
-
-/* Buffer wrangling helpers API. */
-int netfs_alloc_folioq_buffer(struct address_space *mapping,
- struct folio_queue **_buffer,
- size_t *_cur_size, ssize_t size, gfp_t gfp);
-void netfs_free_folioq_buffer(struct folio_queue *fq);
-
/* Writeback exclusion API. */
bool netfs_wb_begin(struct netfs_inode *ictx, bool nowait);
void netfs_wb_end(struct netfs_inode *ictx);
diff --git a/include/linux/rolling_buffer.h b/include/linux/rolling_buffer.h
index a97f7cfaacaad..5c0bc4221f010 100644
--- a/include/linux/rolling_buffer.h
+++ b/include/linux/rolling_buffer.h
@@ -8,49 +8,35 @@
#ifndef _ROLLING_BUFFER_H
#define _ROLLING_BUFFER_H
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
#include <linux/uio.h>
/*
- * Rolling buffer. Whilst the buffer is live and in use, folios and folio
- * queue segments can be added to one end by one thread and removed from the
- * other end by another thread. The buffer isn't allowed to be empty; it must
- * always have at least one folio_queue in it so that neither side has to
- * modify both queue pointers.
+ * Rolling buffer. Whilst the buffer is live and in use, folios and bvecq
+ * segments can be added to one end by one thread and removed from the other
+ * end by another thread. The buffer isn't allowed to be empty; it must always
+ * have at least one bvecq in it so that neither side has to modify both queue
+ * pointers.
*
* The iterator in the buffer is extended as buffers are inserted. It can be
* snapshotted to use a segment of the buffer.
*/
struct rolling_buffer {
- struct folio_queue *head; /* Producer's insertion point */
- struct folio_queue *tail; /* Consumer's removal point */
+ struct bvecq *head; /* Producer's insertion point */
+ struct bvecq *tail; /* Consumer's removal point */
struct iov_iter iter; /* Iterator tracking what's left in the buffer */
- u8 next_head_slot; /* Next slot in ->head */
u8 first_tail_slot; /* First slot in ->tail */
+ bool for_writeback; /* T if being used for writeback */
};
-/*
- * Snapshot of a rolling buffer.
- */
-struct rolling_buffer_snapshot {
- struct folio_queue *curr_folioq; /* Queue segment in which current folio resides */
- unsigned char curr_slot; /* Folio currently being read */
- unsigned char curr_order; /* Order of folio */
-};
-
-/* Marks to store per-folio in the internal folio_queue structs. */
-#define ROLLBUF_MARK_1 BIT(0)
-#define ROLLBUF_MARK_2 BIT(1)
-
-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,
- unsigned int direction, gfp_t gfp);
+int rolling_buffer_init(struct rolling_buffer *roll, unsigned int direction,
+ gfp_t gfp, bool for_writeback);
int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp);
ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
struct readahead_control *ractl,
- unsigned int rreq_id, gfp_t gfp);
-ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
- unsigned int flags, gfp_t gfp);
-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll);
+ gfp_t gfp);
+ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio, gfp_t gfp);
+struct bvecq *rolling_buffer_delete_spent(struct rolling_buffer *roll);
void rolling_buffer_clear(struct rolling_buffer *roll);
static inline void rolling_buffer_advance(struct rolling_buffer *roll, size_t amount)
diff --git a/include/linux/uio.h b/include/linux/uio.h
index fe2e985d74d24..8d82a8258ef11 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -11,7 +11,6 @@
#include <uapi/linux/uio.h>
struct page;
-struct folio_queue;
typedef unsigned int __bitwise iov_iter_extraction_t;
@@ -26,7 +25,7 @@ enum iter_type {
ITER_IOVEC,
ITER_BVEC,
ITER_KVEC,
- ITER_FOLIOQ,
+ ITER_BVECQ,
ITER_XARRAY,
ITER_DISCARD,
};
@@ -68,7 +67,7 @@ struct iov_iter {
const struct iovec *__iov;
const struct kvec *kvec;
const struct bio_vec *bvec;
- const struct folio_queue *folioq;
+ const struct bvecq *bvecq;
struct xarray *xarray;
void __user *ubuf;
};
@@ -77,7 +76,7 @@ struct iov_iter {
};
union {
unsigned long nr_segs;
- u8 folioq_slot;
+ u16 bvecq_slot;
loff_t xarray_start;
};
};
@@ -145,9 +144,9 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
return iov_iter_type(i) == ITER_DISCARD;
}
-static inline bool iov_iter_is_folioq(const struct iov_iter *i)
+static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
{
- return iov_iter_type(i) == ITER_FOLIOQ;
+ return iov_iter_type(i) == ITER_BVECQ;
}
static inline bool iov_iter_is_xarray(const struct iov_iter *i)
@@ -295,9 +294,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec
void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec,
unsigned long nr_segs, size_t count);
void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
- const struct folio_queue *folioq,
- unsigned int first_slot, unsigned int offset, size_t count);
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq,
+ unsigned int first_slot, unsigned int offset, size_t count);
void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xarray *xarray,
loff_t start, size_t count);
ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index bf1e1f185b05c..1f92a653d3bfa 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -244,14 +244,6 @@
EM(netfs_trace_donate_to_next, "to-next") \
E_(netfs_trace_donate_to_deferred_next, "defer-next")
-#define netfs_folioq_traces \
- EM(netfs_trace_folioq_alloc_buffer, "alloc-buf") \
- EM(netfs_trace_folioq_clear, "clear") \
- EM(netfs_trace_folioq_delete, "delete") \
- EM(netfs_trace_folioq_make_space, "make-space") \
- EM(netfs_trace_folioq_rollbuf_init, "roll-init") \
- E_(netfs_trace_folioq_read_progress, "r-progress")
-
#ifndef __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
#define __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
@@ -270,7 +262,6 @@ enum netfs_sreq_ref_trace { netfs_sreq_ref_traces } __mode(byte);
enum netfs_folio_trace { netfs_folio_traces } __mode(byte);
enum netfs_collect_contig_trace { netfs_collect_contig_traces } __mode(byte);
enum netfs_donate_trace { netfs_donate_traces } __mode(byte);
-enum netfs_folioq_trace { netfs_folioq_traces } __mode(byte);
#endif
@@ -294,7 +285,6 @@ netfs_sreq_ref_traces;
netfs_folio_traces;
netfs_collect_contig_traces;
netfs_donate_traces;
-netfs_folioq_traces;
/*
* Now redefine the EM() and E_() macros to map the enums to the strings that
@@ -398,7 +388,7 @@ TRACE_EVENT(netfs_sreq,
__entry->len = sreq->len;
__entry->transferred = sreq->transferred;
__entry->start = sreq->start;
- __entry->slot = sreq->io_iter.folioq_slot;
+ __entry->slot = sreq->io_iter.bvecq_slot;
),
TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx s=%u e=%d",
@@ -773,29 +763,6 @@ TRACE_EVENT(netfs_collect_stream,
__entry->collected_to, __entry->issued_to)
);
-TRACE_EVENT(netfs_folioq,
- TP_PROTO(const struct folio_queue *fq,
- enum netfs_folioq_trace trace),
-
- TP_ARGS(fq, trace),
-
- TP_STRUCT__entry(
- __field(unsigned int, rreq)
- __field(unsigned int, id)
- __field(enum netfs_folioq_trace, trace)
- ),
-
- TP_fast_assign(
- __entry->rreq = fq ? fq->rreq_id : 0;
- __entry->id = fq ? fq->debug_id : 0;
- __entry->trace = trace;
- ),
-
- TP_printk("R=%08x fq=%x %s",
- __entry->rreq, __entry->id,
- __print_symbolic(__entry->trace, netfs_folioq_traces))
- );
-
TRACE_EVENT(netfs_read_progress_at,
TP_PROTO(const struct netfs_io_request *rreq),
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index d870bc5e50bc8..63fd6156ffa7c 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -6803,8 +6803,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {
/* amdgpu */
{ "amdgpu_vm_bo_map", 0x1 },
{ "amdgpu_vm_bo_unmap", 0x1 },
- /* netfs */
- { "netfs_folioq", 0x1 },
/* xfs from xfs_defer_pending_class */
{ "xfs_defer_create_intent", 0x1 },
{ "xfs_defer_cancel_list", 0x1 },
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 2072c04e99d05..7dcb81ba2a73a 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -538,37 +538,38 @@ static void iov_iter_iovec_advance(struct iov_iter *i, size_t size)
i->__iov = iov;
}
-static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)
+static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)
{
- const struct folio_queue *folioq = i->folioq;
- unsigned int slot = i->folioq_slot;
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
if (!i->count)
return;
- i->count -= size;
-
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- slot = 0;
- }
+ i->count -= by;
- size += i->iov_offset; /* From beginning of current segment. */
+ by += i->iov_offset; /* From beginning of current segment. */
do {
- size_t fsize = folioq_folio_size(folioq, slot);
+ size_t len;
- if (likely(size < fsize))
- break;
- size -= fsize;
- slot++;
- if (slot >= folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
+ if (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
slot = 0;
+ continue;
}
- } while (size);
- i->iov_offset = size;
- i->folioq_slot = slot;
- i->folioq = folioq;
+ len = bq->bv[slot].bv_len;
+
+ if (likely(by < len))
+ break;
+ by -= len;
+ slot++;
+ } while (by);
+
+ i->iov_offset = by;
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
}
void iov_iter_advance(struct iov_iter *i, size_t size)
@@ -583,38 +584,39 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
iov_iter_iovec_advance(i, size);
} else if (iov_iter_is_bvec(i)) {
iov_iter_bvec_advance(i, size);
- } else if (iov_iter_is_folioq(i)) {
- iov_iter_folioq_advance(i, size);
+ } else if (iov_iter_is_bvecq(i)) {
+ iov_iter_bvecq_advance(i, size);
} else if (iov_iter_is_discard(i)) {
i->count -= size;
}
}
EXPORT_SYMBOL(iov_iter_advance);
-static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)
+static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)
{
- const struct folio_queue *folioq = i->folioq;
- unsigned int slot = i->folioq_slot;
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
for (;;) {
- size_t fsize;
+ size_t len;
if (slot == 0) {
- folioq = folioq->prev;
- slot = folioq_nr_slots(folioq);
+ bq = bq->prev;
+ slot = bq->nr_slots;
+ continue;
}
slot--;
- fsize = folioq_folio_size(folioq, slot);
- if (unroll <= fsize) {
- i->iov_offset = fsize - unroll;
+ len = bq->bv[slot].bv_len;
+ if (unroll <= len) {
+ i->iov_offset = len - unroll;
break;
}
- unroll -= fsize;
+ unroll -= len;
}
- i->folioq_slot = slot;
- i->folioq = folioq;
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
}
void iov_iter_revert(struct iov_iter *i, size_t unroll)
@@ -648,9 +650,9 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
}
unroll -= n;
}
- } else if (iov_iter_is_folioq(i)) {
+ } else if (iov_iter_is_bvecq(i)) {
i->iov_offset = 0;
- iov_iter_folioq_revert(i, unroll);
+ iov_iter_bvecq_revert(i, unroll);
} else { /* same logics for iovec and kvec */
const struct iovec *iov = iter_iov(i);
while (1) {
@@ -678,9 +680,28 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return min(i->count, i->bvec->bv_len - i->iov_offset);
}
- if (unlikely(iov_iter_is_folioq(i)))
- return !i->count ? 0 :
- umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
+ if (!i->count)
+ return 0;
+ if (unlikely(iov_iter_is_bvecq(i))) {
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+ size_t offset = i->iov_offset;
+
+ for (;;) {
+ while (slot >= bq->nr_slots) {
+ bq = bq->next;
+ if (!bq)
+ return 0;
+ slot = 0;
+ offset = 0;
+ }
+ if (bq->bv[slot].bv_len > offset)
+ break;
+ slot++;
+ offset = 0;
+ }
+ return min(i->count, bq->bv[slot].bv_len - offset);
+ }
return i->count;
}
EXPORT_SYMBOL(iov_iter_single_seg_count);
@@ -718,34 +739,33 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction,
EXPORT_SYMBOL(iov_iter_bvec);
/**
- * iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue
+ * iov_iter_bvec_queue - Initialise an I/O iterator to use a segmented bvec queue
* @i: The iterator to initialise.
* @direction: The direction of the transfer.
- * @folioq: The starting point in the folio queue.
- * @first_slot: The first slot in the folio queue to use
- * @offset: The offset into the folio in the first slot to start at
+ * @bvecq: The starting point in the bvec queue.
+ * @first_slot: The first slot in the bvec queue to use
+ * @offset: The offset into the bvec in the first slot to start at
* @count: The size of the I/O buffer in bytes.
*
- * Set up an I/O iterator to either draw data out of the pages attached to an
- * inode or to inject data into those pages. The pages *must* be prevented
- * from evaporation, either by taking a ref on them or locking them by the
- * caller.
+ * Set up an I/O iterator to either draw data out of the buffers attached to an
+ * inode or to inject data into those buffers. The pages *must* be prevented
+ * from evaporation, either by the caller.
*/
-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
- const struct folio_queue *folioq, unsigned int first_slot,
- unsigned int offset, size_t count)
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq, unsigned int first_slot,
+ unsigned int offset, size_t count)
{
- BUG_ON(direction & ~1);
+ WARN_ON(direction & ~(READ | WRITE));
*i = (struct iov_iter) {
- .iter_type = ITER_FOLIOQ,
- .data_source = direction,
- .folioq = folioq,
- .folioq_slot = first_slot,
- .count = count,
- .iov_offset = offset,
+ .iter_type = ITER_BVECQ,
+ .data_source = direction,
+ .bvecq = bvecq,
+ .bvecq_slot = first_slot,
+ .count = count,
+ .iov_offset = offset,
};
}
-EXPORT_SYMBOL(iov_iter_folio_queue);
+EXPORT_SYMBOL(iov_iter_bvec_queue);
/**
* iov_iter_xarray - Initialise an I/O iterator to use the pages in an xarray
@@ -839,6 +859,39 @@ static unsigned long iov_iter_alignment_bvec(const struct iov_iter *i)
return res;
}
+static unsigned long iov_iter_alignment_bvecq(const struct iov_iter *iter)
+{
+ const struct bvecq *bq;
+ unsigned long res = 0;
+ unsigned int slot = iter->bvecq_slot;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ if (!size)
+ return res;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t part = min(bvec->bv_len - skip, size);
+
+ if (part) {
+ res |= bvec->bv_offset + skip;
+ res |= part;
+ }
+
+ size -= part;
+ if (size == 0)
+ return res;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+
+ return res;
+}
+
unsigned long iov_iter_alignment(const struct iov_iter *i)
{
if (likely(iter_is_ubuf(i))) {
@@ -854,10 +907,10 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return iov_iter_alignment_bvec(i);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_alignment_bvecq(i);
- /* With both xarray and folioq types, we're dealing with whole folios. */
- if (iov_iter_is_folioq(i))
- return i->iov_offset | i->count;
+ /* With the xarray type, we're dealing with whole folios. */
if (iov_iter_is_xarray(i))
return (i->xarray_start + i->iov_offset) | i->count;
@@ -910,62 +963,156 @@ static int want_pages_array(struct page ***res, size_t size,
return count;
}
-static ssize_t iter_folioq_get_pages(struct iov_iter *iter,
- struct page ***ppages, size_t maxsize,
- unsigned maxpages, size_t *_start_offset)
+/*
+ * Count the number of virtually contiguous pages coming up next in an
+ * ITER_BVECQ iterator, up to the specified maxima.
+ */
+static unsigned int iter_count_bvecq_pages(const struct iov_iter *iter,
+ size_t maxsize,
+ unsigned int maxpages)
{
- const struct folio_queue *folioq = iter->folioq;
+ const struct bvecq *bvecq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
+ ssize_t remain = umin(maxsize, iter->count);
+ size_t count = 0, offset = iter->iov_offset;
+
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
+
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(remain > 0);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+
+ bv = &bvecq->bv[slot++];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ /* bv_page is not allowed to be NULL unless bv_len == 0. */
+ if (WARN_ON_ONCE(!bv->bv_page && blen > 0))
+ break;
+ if (!PAGE_ALIGNED(boff) && count > 0)
+ break;
+
+ boff += offset;
+ blen -= offset;
+ offset = 0;
+ if (!blen)
+ continue;
+
+ blen = umin(blen, remain);
+ remain -= blen;
+ blen += offset_in_page(boff);
+ count += DIV_ROUND_UP(blen, PAGE_SIZE);
+
+ if (!PAGE_ALIGNED(blen))
+ break;
+ } while (remain > 0 && count < maxpages);
+
+ return min(count, maxpages);
+}
+
+/*
+ * Get a list of virtually contiguous pages from an ITER_BVECQ iterator and pin
+ * them.
+ */
+static ssize_t iter_bvecq_get_pages(struct iov_iter *iter,
+ struct page ***ppages, size_t maxsize,
+ unsigned int maxpages, size_t *_start_offset)
+{
+ const struct bvecq *bvecq = iter->bvecq;
struct page **pages;
- unsigned int slot = iter->folioq_slot;
- size_t extracted = 0, count = iter->count, iov_offset = iter->iov_offset;
+ unsigned int slot = iter->bvecq_slot, nr = 0;
+ size_t extracted = 0, offset = iter->iov_offset;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- slot = 0;
- if (WARN_ON(iov_offset != 0))
- return -EIO;
+ for (;;) {
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next)
+ return 0;
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+ if (offset >= bvecq->bv[slot].bv_len) {
+ slot++;
+ offset = 0;
+ continue;
+ }
+ break;
}
- maxpages = want_pages_array(ppages, maxsize, iov_offset & ~PAGE_MASK, maxpages);
+ /* Count the next run of virtually contiguous pages. */
+ maxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);
+ if (!maxpages)
+ return 0;
+
+ maxpages = want_pages_array(ppages, maxsize,
+ (bvecq->bv[slot].bv_offset + offset) & ~PAGE_MASK,
+ maxpages);
if (!maxpages)
return -ENOMEM;
- *_start_offset = iov_offset & ~PAGE_MASK;
pages = *ppages;
- for (;;) {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t offset = iov_offset, fsize = folioq_folio_size(folioq, slot);
- size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
- if (offset < fsize) {
- part = umin(part, umin(maxsize - extracted, fsize - offset));
- count -= part;
- iov_offset += part;
- extracted += part;
+ /* Now transcribe the page pointers. */
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
- *pages = folio_page(folio, offset / PAGE_SIZE);
- get_page(*pages);
- pages++;
- maxpages--;
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(extracted < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
}
- if (maxpages == 0 || extracted >= maxsize)
- break;
+ bv = &bvecq->bv[slot];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ /* bv_page is not allowed to be NULL unless bv_len == 0. */
+
+ if (offset < blen) {
+ size_t poff = (boff + offset) % PAGE_SIZE;
+ size_t part = min(maxsize - extracted, blen - offset);
+ size_t pix = (boff + offset) / PAGE_SIZE;
+
+ if (poff + part > PAGE_SIZE)
+ part = PAGE_SIZE - poff;
+
+ if (!extracted)
+ *_start_offset = poff;
- if (iov_offset >= fsize) {
- iov_offset = 0;
+ /* Get a ref on the page if it's not a slab page. */
+ if (!PageSlab(bv->bv_page + pix))
+ get_page(bv->bv_page + pix);
+
+ pages[nr++] = bv->bv_page + pix;
+ offset += part;
+ extracted += part;
+ }
+
+ if (offset >= blen) {
+ offset = 0;
slot++;
- if (slot == folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
- slot = 0;
- }
}
- }
+ } while (nr < maxpages && extracted < maxsize);
- iter->count = count;
- iter->iov_offset = iov_offset;
- iter->folioq = folioq;
- iter->folioq_slot = slot;
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= extracted;
return extracted;
}
@@ -1120,8 +1267,8 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
}
return maxsize;
}
- if (iov_iter_is_folioq(i))
- return iter_folioq_get_pages(i, pages, maxsize, maxpages, start);
+ if (iov_iter_is_bvecq(i))
+ return iter_bvecq_get_pages(i, pages, maxsize, maxpages, start);
if (iov_iter_is_xarray(i))
return iter_xarray_get_pages(i, pages, maxsize, maxpages, start);
return -EFAULT;
@@ -1192,6 +1339,38 @@ static int bvec_npages(const struct iov_iter *i, int maxpages)
return npages;
}
+static size_t iov_npages_bvecq(const struct iov_iter *iter, size_t maxpages)
+{
+ const struct bvecq *bq;
+ unsigned int slot = iter->bvecq_slot;
+ size_t npages = 0;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t offs = (bvec->bv_offset + skip) % PAGE_SIZE;
+ size_t part = min(bvec->bv_len - skip, size);
+
+ if (part) {
+ npages += DIV_ROUND_UP(offs + part, PAGE_SIZE);
+ if (npages >= maxpages)
+ goto out;
+ }
+
+ size -= part;
+ if (!size)
+ goto out;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+out:
+ return umin(npages, maxpages);
+}
+
int iov_iter_npages(const struct iov_iter *i, int maxpages)
{
if (unlikely(!i->count))
@@ -1206,11 +1385,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
return iov_npages(i, maxpages);
if (iov_iter_is_bvec(i))
return bvec_npages(i, maxpages);
- if (iov_iter_is_folioq(i)) {
- unsigned offset = i->iov_offset % PAGE_SIZE;
- int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
- return min(npages, maxpages);
- }
+ if (iov_iter_is_bvecq(i))
+ return iov_npages_bvecq(i, maxpages);
if (iov_iter_is_xarray(i)) {
unsigned offset = (i->xarray_start + i->iov_offset) % PAGE_SIZE;
int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
@@ -1494,64 +1670,86 @@ void iov_iter_restore(struct iov_iter *i, struct iov_iter_state *state)
EXPORT_SYMBOL_FOR_MODULES(iov_iter_restore, "vmw_vsock_virtio_transport_common");
/*
- * Extract a list of contiguous pages from an ITER_FOLIOQ iterator. This does
- * not get references on the pages, nor does it get a pin on them.
+ * Extract a list of virtually contiguous pages from an ITER_BVECQ iterator.
+ * This does not get references on the pages, nor does it get a pin on them.
*/
-static ssize_t iov_iter_extract_folioq_pages(struct iov_iter *i,
- struct page ***pages, size_t maxsize,
- unsigned int maxpages,
- iov_iter_extraction_t extraction_flags,
- size_t *offset0)
+static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,
+ struct page ***pages, size_t maxsize,
+ unsigned int maxpages,
+ iov_iter_extraction_t extraction_flags,
+ size_t *offset0)
{
- const struct folio_queue *folioq = i->folioq;
+ const struct bvecq *bvecq;
struct page **p;
- unsigned int nr = 0;
- size_t extracted = 0, offset, slot = i->folioq_slot;
+ unsigned int slot, nr = 0;
+ size_t extracted = 0, offset;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- slot = 0;
- if (WARN_ON(i->iov_offset != 0))
- return -EIO;
- }
+ /* Count the next run of virtually contiguous pages. */
+ maxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);
+ if (!maxpages)
+ return 0;
- offset = i->iov_offset & ~PAGE_MASK;
- *offset0 = offset;
+ if (!*pages) {
+ *pages = kvmalloc_array(maxpages, sizeof(struct page *), GFP_KERNEL);
+ if (!*pages)
+ return -ENOMEM;
+ }
- maxpages = want_pages_array(pages, maxsize, offset, maxpages);
- if (!maxpages)
- return -ENOMEM;
p = *pages;
- for (;;) {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t offset = i->iov_offset, fsize = folioq_folio_size(folioq, slot);
- size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
- if (offset < fsize) {
- part = umin(part, umin(maxsize - extracted, fsize - offset));
- i->count -= part;
- i->iov_offset += part;
- extracted += part;
+ /* Now transcribe the page pointers. */
+ extracted = 0;
+ bvecq = iter->bvecq;
+ offset = iter->iov_offset;
+ slot = iter->bvecq_slot;
+
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
- p[nr++] = folio_page(folio, offset / PAGE_SIZE);
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(extracted < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
}
- if (nr >= maxpages || extracted >= maxsize)
- break;
+ bv = &bvecq->bv[slot];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
- if (i->iov_offset >= fsize) {
- i->iov_offset = 0;
+ /* bv_page is not allowed to be NULL unless bv_len == 0. */
+
+ if (offset < blen) {
+ size_t part = umin(maxsize - extracted, blen - offset);
+ size_t poff = (boff + offset) % PAGE_SIZE;
+ size_t pix = (boff + offset) / PAGE_SIZE;
+
+ if (poff + part > PAGE_SIZE)
+ part = PAGE_SIZE - poff;
+
+ if (!extracted)
+ *offset0 = poff;
+
+ p[nr++] = bv->bv_page + pix;
+ offset += part;
+ extracted += part;
+ }
+
+ if (offset >= blen) {
+ offset = 0;
slot++;
- if (slot == folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
- slot = 0;
- }
}
- }
+ } while (nr < maxpages && extracted < maxsize);
- i->folioq = folioq;
- i->folioq_slot = slot;
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= extracted;
return extracted;
}
@@ -1817,8 +2015,8 @@ static ssize_t iov_iter_extract_user_pages(struct iov_iter *i,
* added to the pages, but refs will not be taken.
* iov_iter_extract_will_pin() will return true.
*
- * (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_FOLIOQ or ITER_XARRAY, the
- * pages are merely listed; no extra refs or pins are obtained.
+ * (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_XARRAY, the pages are
+ * merely listed; no extra refs or pins are obtained.
* iov_iter_extract_will_pin() will return 0.
*
* Note also:
@@ -1853,10 +2051,10 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,
return iov_iter_extract_bvec_pages(i, pages, maxsize,
maxpages, extraction_flags,
offset0);
- if (iov_iter_is_folioq(i))
- return iov_iter_extract_folioq_pages(i, pages, maxsize,
- maxpages, extraction_flags,
- offset0);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_extract_bvecq_pages(i, pages, maxsize,
+ maxpages, extraction_flags,
+ offset0);
if (iov_iter_is_xarray(i))
return iov_iter_extract_xarray_pages(i, pages, maxsize,
maxpages, extraction_flags,
diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index 6ea40d2e62475..b9a7298306d95 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -10,8 +10,8 @@
#include <linux/highmem.h>
#include <linux/kmemleak.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/uio.h>
-#include <linux/folio_queue.h>
/**
* sg_nents - return total count of entries in scatterlist
@@ -1268,61 +1268,59 @@ static ssize_t extract_kvec_to_sg(struct iov_iter *iter,
}
/*
- * Extract up to sg_max folios from an FOLIOQ-type iterator and add them to
+ * Extract up to sg_max folios from an BVECQ-type iterator and add them to
* the scatterlist. The pages are not pinned.
*/
-static ssize_t extract_folioq_to_sg(struct iov_iter *iter,
+static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,
ssize_t maxsize,
struct sg_table *sgtable,
unsigned int sg_max,
iov_iter_extraction_t extraction_flags)
{
- const struct folio_queue *folioq = iter->folioq;
+ const struct bvecq *bvecq = iter->bvecq;
struct scatterlist *sg = sgtable->sgl + sgtable->nents;
- unsigned int slot = iter->folioq_slot;
+ unsigned int slot = iter->bvecq_slot;
ssize_t ret = 0;
size_t offset = iter->iov_offset;
- BUG_ON(!folioq);
+ maxsize = umin(maxsize, iov_iter_count(iter));
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- if (WARN_ON_ONCE(!folioq))
- return 0;
- slot = 0;
- }
-
- do {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t fsize = folioq_folio_size(folioq, slot);
+ while (sg_max > 0 && ret < maxsize) {
+ const struct bio_vec *bv;
+ size_t blen, part;
- if (offset < fsize) {
- size_t part = umin(maxsize - ret, fsize - offset);
-
- sg_set_page(sg, folio_page(folio, 0), part, offset);
- sgtable->nents++;
- sg++;
- sg_max--;
- offset += part;
- ret += part;
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(ret < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
}
- if (offset >= fsize) {
+ bv = &bvecq->bv[slot];
+ blen = bv->bv_len;
+
+ if (offset >= blen) {
offset = 0;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- if (!folioq->next) {
- WARN_ON_ONCE(ret < iter->count);
- break;
- }
- folioq = folioq->next;
- slot = 0;
- }
+ continue;
}
- } while (sg_max > 0 && ret < maxsize);
- iter->folioq = folioq;
- iter->folioq_slot = slot;
+ part = umin(maxsize - ret, blen - offset);
+
+ sg_set_page(sg, bv->bv_page, part, bv->bv_offset + offset);
+ sgtable->nents++;
+ sg++;
+ sg_max--;
+ offset += part;
+ ret += part;
+ }
+
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
iter->iov_offset = offset;
iter->count -= ret;
return ret;
@@ -1391,8 +1389,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,
* addition of @sg_max elements.
*
* The pages referred to by UBUF- and IOVEC-type iterators are extracted and
- * pinned; BVEC-, KVEC-, FOLIOQ- and XARRAY-type are extracted but aren't
- * pinned; DISCARD-type is not supported.
+ * pinned; BVEC-, BVECQ-, KVEC-, XARRAY-type are extracted but aren't pinned;
+ * DISCARD-type is not supported.
*
* No end mark is placed on the scatterlist; that's left to the caller.
*
@@ -1424,9 +1422,9 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,
case ITER_KVEC:
return extract_kvec_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
- case ITER_FOLIOQ:
- return extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,
- extraction_flags);
+ case ITER_BVECQ:
+ return extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,
+ extraction_flags);
case ITER_XARRAY:
return extract_xarray_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
diff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c
index 32e42d8c7ca13..a2a83ba613fda 100644
--- a/lib/tests/kunit_iov_iter.c
+++ b/lib/tests/kunit_iov_iter.c
@@ -12,7 +12,7 @@
#include <linux/mm.h>
#include <linux/uio.h>
#include <linux/bvec.h>
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
#include <linux/scatterlist.h>
#include <linux/minmax.h>
#include <linux/mman.h>
@@ -382,58 +382,67 @@ static void __init iov_kunit_copy_from_bvec(struct kunit *test)
KUNIT_SUCCEED(test);
}
-static void iov_kunit_destroy_folioq(void *data)
+static void iov_kunit_destroy_bvecq(void *data)
{
- struct folio_queue *folioq, *next;
+ struct bvecq *bq, *next;
- for (folioq = data; folioq; folioq = next) {
- next = folioq->next;
- kfree(folioq);
+ for (bq = data; bq; bq = next) {
+ next = bq->next;
+ /* The pages are freed by vmap with VM_MAP_PUT_PAGES. */
+ kfree(bq);
}
}
-static void __init iov_kunit_load_folioq(struct kunit *test,
+static struct bvecq *iov_kunit_alloc_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = kzalloc(struct_size(bq, __bv, max_slots), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, bq);
+ bq->max_slots = max_slots;
+ bq->bv = bq->__bv;
+ bq->inline_bv = true;
+ return bq;
+}
+
+static struct bvecq *iov_kunit_create_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = iov_kunit_alloc_bvecq(test, max_slots);
+ kunit_add_action_or_reset(test, iov_kunit_destroy_bvecq, bq);
+ return bq;
+}
+
+static void __init iov_kunit_load_bvecq(struct kunit *test,
struct iov_iter *iter, int dir,
- struct folio_queue *folioq,
+ struct bvecq *bq_head,
struct page **pages, size_t npages)
{
- struct folio_queue *p = folioq;
+ struct bvecq *bq = bq_head;
size_t size = 0;
- int i;
- for (i = 0; i < npages; i++) {
- if (folioq_full(p)) {
- p->next = kzalloc_obj(struct folio_queue);
- KUNIT_ASSERT_NOT_ERR_OR_NULL(test, p->next);
- folioq_init(p->next, 0);
- p->next->prev = p;
- p = p->next;
+ for (int i = 0; i < npages; i++) {
+ if (bq->nr_slots >= bq->max_slots) {
+ bq->next = iov_kunit_alloc_bvecq(test, 13);
+ bq->next->prev = bq;
+ bq = bq->next;
}
- folioq_append(p, page_folio(pages[i]));
+ bvec_set_page(&bq->bv[bq->nr_slots], pages[i], PAGE_SIZE, 0);
+ bq->nr_slots++;
size += PAGE_SIZE;
}
- iov_iter_folio_queue(iter, dir, folioq, 0, 0, size);
-}
-
-static struct folio_queue *iov_kunit_create_folioq(struct kunit *test)
-{
- struct folio_queue *folioq;
-
- folioq = kzalloc_obj(struct folio_queue);
- KUNIT_ASSERT_NOT_ERR_OR_NULL(test, folioq);
- kunit_add_action_or_reset(test, iov_kunit_destroy_folioq, folioq);
- folioq_init(folioq, 0);
- return folioq;
+ iov_iter_bvec_queue(iter, dir, bq_head, 0, 0, size);
}
/*
- * Test copying to a ITER_FOLIOQ-type iterator.
+ * Test copying to a ITER_BVECQ-type iterator.
*/
-static void __init iov_kunit_copy_to_folioq(struct kunit *test)
+static void __init iov_kunit_copy_to_bvecq(struct kunit *test)
{
const struct kvec_test_range *pr;
struct iov_iter iter;
- struct folio_queue *folioq;
+ struct bvecq *bq;
struct page **spages, **bpages;
u8 *scratch, *buffer;
size_t bufsize, npages, size, copied;
@@ -442,7 +451,7 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)
bufsize = 0x100000;
npages = bufsize / PAGE_SIZE;
- folioq = iov_kunit_create_folioq(test);
+ bq = iov_kunit_create_bvecq(test, 13);
scratch = iov_kunit_create_buffer(test, &spages, npages);
for (i = 0; i < bufsize; i++)
@@ -451,20 +460,19 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)
buffer = iov_kunit_create_buffer(test, &bpages, npages);
memset(buffer, 0, bufsize);
- iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
i = 0;
for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
size = pr->to - pr->from;
KUNIT_ASSERT_LE(test, pr->to, bufsize);
- iov_iter_folio_queue(&iter, READ, folioq, 0, 0, pr->to);
+ iov_iter_bvec_queue(&iter, READ, bq, 0, 0, pr->to);
iov_iter_advance(&iter, pr->from);
copied = copy_to_iter(scratch + i, size, &iter);
KUNIT_EXPECT_EQ(test, copied, size);
KUNIT_EXPECT_EQ(test, iter.count, 0);
- KUNIT_EXPECT_EQ(test, iter.iov_offset, pr->to % PAGE_SIZE);
i += size;
if (test->status == KUNIT_FAILURE)
goto stop;
@@ -489,13 +497,13 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)
}
/*
- * Test copying from a ITER_FOLIOQ-type iterator.
+ * Test copying from a ITER_BVECQ-type iterator.
*/
-static void __init iov_kunit_copy_from_folioq(struct kunit *test)
+static void __init iov_kunit_copy_from_bvecq(struct kunit *test)
{
const struct kvec_test_range *pr;
struct iov_iter iter;
- struct folio_queue *folioq;
+ struct bvecq *bq;
struct page **spages, **bpages;
u8 *scratch, *buffer;
size_t bufsize, npages, size, copied;
@@ -504,7 +512,7 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)
bufsize = 0x100000;
npages = bufsize / PAGE_SIZE;
- folioq = iov_kunit_create_folioq(test);
+ bq = iov_kunit_create_bvecq(test, 13);
buffer = iov_kunit_create_buffer(test, &bpages, npages);
for (i = 0; i < bufsize; i++)
@@ -513,20 +521,19 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)
scratch = iov_kunit_create_buffer(test, &spages, npages);
memset(scratch, 0, bufsize);
- iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
i = 0;
for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
size = pr->to - pr->from;
KUNIT_ASSERT_LE(test, pr->to, bufsize);
- iov_iter_folio_queue(&iter, WRITE, folioq, 0, 0, pr->to);
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
iov_iter_advance(&iter, pr->from);
copied = copy_from_iter(scratch + i, size, &iter);
KUNIT_EXPECT_EQ(test, copied, size);
KUNIT_EXPECT_EQ(test, iter.count, 0);
- KUNIT_EXPECT_EQ(test, iter.iov_offset, pr->to % PAGE_SIZE);
i += size;
}
@@ -868,13 +875,13 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)
}
/*
- * Test the extraction of ITER_FOLIOQ-type iterators.
+ * Test the extraction of ITER_BVECQ-type iterators.
*/
-static void __init iov_kunit_extract_pages_folioq(struct kunit *test)
+static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)
{
const struct kvec_test_range *pr;
- struct folio_queue *folioq;
struct iov_iter iter;
+ struct bvecq *bq;
struct page **bpages, *pagelist[8], **pages = pagelist;
ssize_t len;
size_t bufsize, size = 0, npages;
@@ -883,17 +890,17 @@ static void __init iov_kunit_extract_pages_folioq(struct kunit *test)
bufsize = 0x100000;
npages = bufsize / PAGE_SIZE;
- folioq = iov_kunit_create_folioq(test);
+ bq = iov_kunit_create_bvecq(test, 13);
iov_kunit_create_buffer(test, &bpages, npages);
- iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
from = pr->from;
size = pr->to - from;
KUNIT_ASSERT_LE(test, pr->to, bufsize);
- iov_iter_folio_queue(&iter, WRITE, folioq, 0, 0, pr->to);
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
iov_iter_advance(&iter, from);
do {
@@ -1173,23 +1180,6 @@ static void __init iov_kunit_iter_to_sg_bvec(struct kunit *test)
iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
}
-static void __init iov_kunit_iter_to_sg_folioq(struct kunit *test)
-{
- struct iov_kunit_iter_to_sg_data data;
- struct folio_queue *folioq;
- struct iov_iter iter;
- size_t bufsize;
-
- bufsize = 0x200000;
- iov_kunit_iter_to_sg_init(test, bufsize, false, &data);
-
- folioq = iov_kunit_create_folioq(test);
- iov_kunit_load_folioq(test, &iter, READ, folioq, data.pages,
- data.npages);
-
- iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
-}
-
static void __init iov_kunit_iter_to_sg_xarray(struct kunit *test)
{
struct iov_kunit_iter_to_sg_data data;
@@ -1226,17 +1216,16 @@ static struct kunit_case __refdata iov_kunit_cases[] = {
KUNIT_CASE(iov_kunit_copy_from_kvec),
KUNIT_CASE(iov_kunit_copy_to_bvec),
KUNIT_CASE(iov_kunit_copy_from_bvec),
- KUNIT_CASE(iov_kunit_copy_to_folioq),
- KUNIT_CASE(iov_kunit_copy_from_folioq),
+ KUNIT_CASE(iov_kunit_copy_to_bvecq),
+ KUNIT_CASE(iov_kunit_copy_from_bvecq),
KUNIT_CASE(iov_kunit_copy_to_xarray),
KUNIT_CASE(iov_kunit_copy_from_xarray),
KUNIT_CASE(iov_kunit_extract_pages_kvec),
KUNIT_CASE(iov_kunit_extract_pages_bvec),
- KUNIT_CASE(iov_kunit_extract_pages_folioq),
+ KUNIT_CASE(iov_kunit_extract_pages_bvecq),
KUNIT_CASE(iov_kunit_extract_pages_xarray),
KUNIT_CASE(iov_kunit_iter_to_sg_kvec),
KUNIT_CASE(iov_kunit_iter_to_sg_bvec),
- KUNIT_CASE(iov_kunit_iter_to_sg_folioq),
KUNIT_CASE(iov_kunit_iter_to_sg_xarray),
KUNIT_CASE(iov_kunit_iter_to_sg_ubuf),
{}
|
| 3/2 | 2026/09/30 13:44 | llm | patch-evaluator |
1mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 61507 output: 33 thoughts: 1117 Thoughts: **Patch Evaluation for Syzkaller Fuzzing** Okay, let's dive into this patch series. The commit message just says "syz-cluster: applied patch under review," which is a bit generic, so I need to dig deeper. The main goal here is to determine if this refactoring warrants fuzzing with syzkaller. First, I'll look at the files changed. I see `Documentation/core-api/folio_queue.rst` and `include/linux/folio_queue.h` are deleted, which immediately catches my eye, alongside the addition of `include/linux/bvecq.h` and a new `bvecq.c` in `fs/netfs`. From the file list, it looks like a fairly comprehensive change: `fs/afs/*`, `fs/netfs/*`, `fs/smb/*`, `include/linux/*`, `lib/*`, and `kernel/bpf/btf.c`. Aha! This is a substantial refactoring. It looks like `folio_queue` / `ITER_FOLIOQ` are being replaced with a segmented `bvecq` ( `ITER_BVECQ`) across quite a few core subsystems: `iov_iter`, `netfs`, AFS, CIFS/SMB, and scatterlist. That's a lot of touchpoints. Now, let's apply my core triage philosophy. The key questions are reachability and what subsystems are involved. Is `netfs`, `iov_iter`, `cifs`, and `afs` reachable by syzkaller? Absolutely! `netfs` is used by *many* filesystems like afs, smb/cifs, nfs, ceph, erofs, cachefiles, and more. `iov_iter` is a foundational VFS/I/O primitive, so it's practically ubiquitous. I see functions like `iov_iter_bvecq_advance`, `iov_iter_bvecq_revert`, and so on, confirming this is a core piece of the I/O path. `fs/netfs/bvecq.c` contains `bvecq_alloc_one`, `bvecq_alloc_chain`, which points to allocation issues and potentially out-of-bounds access. The files in `fs/netfs/rolling_buffer.c`, `fs/netfs/buffered_read.c`, `fs/netfs/write_issue.c`, `fs/netfs/read_collect.c` seem to be directly modifying the netfs read/write pipelines. Is this reachable in standard syzkaller / QEMU fuzzing setups? Yes, easily. Syzkaller is designed to hit filesystems, and `netfs` is a target through various mount points (cachefiles, afs, cifs, 9p, etc.). Operations using `iov_iter`, splice, vfs read/write, readahead, or even just accessing network filesystems would likely exercise this new code. Finally, I need to double-check `Kconfig` dependencies for `netfs` and `bvecq`, as this could have been made modular. I'll quickly check `fs/netfs/Kconfig` using `grepper` or `read-file` to make sure it's enabled by default or in common configurations. Based on my analysis so far, this looks like a strong candidate for syzkaller fuzzing. The deep reachability and core nature of the changed components increases the chances of this patch series uncovering new bugs. |
| 4/2 | 2026/09/30 13:46 | tool | read-file |
0mArgs:{
"File": "fs/netfs/Kconfig",
"FirstLine": 1,
"LineCount": 40
}Results: {
"Contents": " 1:\t# SPDX-License-Identifier: GPL-2.0-only\n 2:\t\n 3:\tconfig NETFS_SUPPORT\n 4:\t\ttristate\n 5:\t\thelp\n 6:\t\t This option enables support for network filesystems, including\n 7:\t\t helpers for high-level buffered I/O, abstracting out read\n 8:\t\t segmentation, local caching and transparent huge page support.\n 9:\t\n 10:\tconfig NETFS_STATS\n 11:\t\tbool \"Gather statistical information on local caching\"\n 12:\t\tdepends on NETFS_SUPPORT \u0026\u0026 PROC_FS\n 13:\t\thelp\n 14:\t\t This option causes statistical information to be gathered on local\n 15:\t\t caching and exported through file:\n 16:\t\n 17:\t\t\t/proc/fs/fscache/stats\n 18:\t\n 19:\t\t The gathering of statistics adds a certain amount of overhead to\n 20:\t\t execution as there are a quite a few stats gathered, and on a\n 21:\t\t multi-CPU system these may be on cachelines that keep bouncing\n 22:\t\t between CPUs. On the other hand, the stats are very useful for\n 23:\t\t debugging purposes. Saying 'Y' here is recommended.\n 24:\t\n 25:\tconfig NETFS_PGPRIV2\n 26:\t\tbool\n 27:\t\n 28:\tconfig NETFS_DEBUG\n 29:\t\tbool \"Enable dynamic debugging netfslib and FS-Cache\"\n 30:\t\tdepends on NETFS_SUPPORT\n 31:\t\thelp\n 32:\t\t This permits debugging to be dynamically enabled in the local caching\n 33:\t\t management module. If this is set, the debugging output may be\n 34:\t\t enabled by setting bits in /sys/module/netfs/parameters/debug.\n 35:\t\n 36:\tconfig FSCACHE\n 37:\t\tbool \"General filesystem local caching manager\"\n 38:\t\tdepends on NETFS_SUPPORT\n 39:\t\thelp\n 40:\t\t This option enables a generic filesystem caching manager that can be\n"
} |
| 5/2 | 2026/09/30 13:46 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 63955 output: 19 thoughts: 15 Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience: **Investigating Dependencies on NETFS_SUPPORT** Okay, so I need to understand the implications of `NETFS_SUPPORT`. The first step is to figure out what functionality in this system either *depends* on or is *selected* by this configuration option. I need to trace this dependency tree to understand which features become available, or disappear, based on the setting of `NETFS_SUPPORT`. Is it a direct dependency, or something more indirect through a series of intermediate options? Are there any unexpected side effects to consider? Let's dive in and start the investigation. |
| 6/2 | 2026/09/30 13:46 | tool | grepper |
0mArgs:{
"Expression": "select NETFS_SUPPORT"
}Results: {
"Output": "fs/9p/Kconfig=2=config 9P_FS\n--\nfs/9p/Kconfig-4-\tdepends on NET_9P\nfs/9p/Kconfig:5:\tselect NETFS_SUPPORT\nfs/9p/Kconfig-6-\thelp\n--\nfs/afs/Kconfig=2=config AFS_FS\n--\nfs/afs/Kconfig-6-\tselect DNS_RESOLVER\nfs/afs/Kconfig:7:\tselect NETFS_SUPPORT\nfs/afs/Kconfig-8-\tselect CRYPTO_KRB5\n--\nfs/ceph/Kconfig=2=config CEPH_FS\n--\nfs/ceph/Kconfig-5-\tselect CEPH_LIB\nfs/ceph/Kconfig:6:\tselect NETFS_SUPPORT\nfs/ceph/Kconfig-7-\tselect NETFS_PGPRIV2\n--\nfs/nfs/Kconfig=173=config NFS_FSCACHE\n--\nfs/nfs/Kconfig-175-\tdepends on NFS_FS\nfs/nfs/Kconfig:176:\tselect NETFS_SUPPORT\nfs/nfs/Kconfig-177-\tselect NETFS_PGPRIV2\n--\nfs/smb/client/Kconfig=2=config CIFS\n--\nfs/smb/client/Kconfig-20-\tselect OID_REGISTRY\nfs/smb/client/Kconfig:21:\tselect NETFS_SUPPORT\nfs/smb/client/Kconfig-22-\thelp\n--\nnet/9p/Kconfig=6=menuconfig NET_9P\nnet/9p/Kconfig-7-\ttristate \"Plan 9 Resource Sharing Support (9P2000)\"\nnet/9p/Kconfig:8:\tselect NETFS_SUPPORT\nnet/9p/Kconfig-9-\thelp\n"
} |
| 7/2 | 2026/09/30 13:46 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 64580 output: 27 thoughts: 96 Thoughts: Okay, let's see what we have here. **Navigating Filesystem Dependencies** Alright, so I'm seeing that `NETFS_SUPPORT` is a pretty fundamental option, as it's directly tied to a bunch of network filesystems: `9P_FS`, `AFS_FS`, `CEPH_FS`, `NFS_FSCACHE`, `CIFS`, and `NET_9P`. These are all fairly common, so that makes sense. Now, let's look at `lib/iov_iter.c`. It looks like it's *always* built – that's what the `obj-y += iov_iter.o` line in `lib/Makefile` indicates. I'm wondering if there's any conditional build logic in `lib/iov_iter.c` itself, or if this is as straightforward as it seems. Let's dig deeper in `lib/Makefile` and make sure my assumptions are valid. I don't want to miss anything. |
| 8/2 | 2026/09/30 13:46 | tool | grepper |
0mArgs:{
"Expression": "iov_iter\\.o",
"PathPrefix": "lib/"
}Results: {
"Output": "lib/Makefile=55=obj-y += bcd.o sort.o parser.o debug_locks.o random32.o \\\nlib/Makefile-56-\t bust_spinlocks.o kasprintf.o bitmap.o scatterlist.o \\\nlib/Makefile:57:\t list_sort.o uuid.o iov_iter.o clz_ctz.o \\\nlib/Makefile-58-\t bsearch.o find_bit.o llist.o lwq.o memweight.o kfifo.o \\\n--\nlib/tests/Makefile=26=obj-$(CONFIG_HASH_KUNIT_TEST) += test_hash.o\nlib/tests/Makefile:27:obj-$(CONFIG_TEST_IOV_ITER) += kunit_iov_iter.o\nlib/tests/Makefile-28-obj-$(CONFIG_IS_SIGNED_TYPE_KUNIT_TEST) += is_signed_type_kunit.o\n"
} |
| 9/2 | 2026/09/30 13:46 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 65155 output: 32 thoughts: 3401 Thoughts: **My Analysis: WorthFuzzing=true - Complex I/O and Buffer Management Changes** Okay, so I've been presented with a patch that's clearly making significant changes to the kernel's I/O and buffer management, and it's got my attention. `iov_iter.o` and `scatterlist.o` are always in the kernel core, and this patch interacts deeply with those, which is a major red flag for potential issues. The fact that `bvecq.c` and its associated data structure, `struct bvecq`, are now conditional on `CONFIG_NETFS_SUPPORT` initially seemed like it might limit the scope, but the dependencies tell a different story. My initial thought was, "Can this new `iov_iter_bvec_queue` thing be used without `NETFS_SUPPORT` being enabled?" That's a key question about the modularity and potential impact of these changes. I see `bvecq` is *used* in `<linux/bvecq.h>` and `bvecq.c` has these *exported* symbols! I need to dig in and see the full picture, where these functions are getting used. I'm now zeroing in on the specific functions introduced, modified, or exported. It seems like a *lot* of functions across several files have been affected, not just the expected `bvecq` ones. `fs/afs` and `fs/smb` are *selecting* `NETFS_SUPPORT`, making this code far more pervasive than initially suspected. This suggests that the changes are much more impactful than the single file changes might indicate. The list of changes is extensive, touching core I/O iterators, scatterlist manipulation, and the `fs/netfs` subsystem, alongside direct usage in AFS and CIFS. This goes well beyond simple refactoring; this is fundamental restructuring. I am looking at all of these: * New Functions: `bvecq_alloc_one`, `bvecq_put`, `bvecq_expand_buffer`, `iov_iter_bvecq_advance`, `iov_iter_bvecq_revert`, and on and on. * Modified Functions: `iov_iter_single_seg_count`, `iov_iter_bvec_queue`, `extract_bvecq_to_sg`... * and new ones in `fs/netfs`, `fs/afs`, and `fs/smb`. I'm looking for bugs and memory-related issues. The `bvecq_alloc_buffer2` function caught my eye immediately. The `pages` pointer arithmetic in `bvecq_alloc_buffer2` looks *very* suspect. They're trying to overlay `pages` (bulk allocated pages) with a pre-allocated part of `p->bv`, which has `struct bio_vec` elements. The pointer manipulation and potential memory overwrites are a definite concern. The potential for reading past the end or writing over the data from a prior allocation makes this a prime target for fuzzing. It's brand new and complex, and this is where vulnerabilities like this hide. The sheer number of `WARN_ON_ONCE` statements is also a huge red flag. Defensive checks, assertions, and potential data corruption warnings all scream "fuzz me!" These assertions are a good start to help fuzzers narrow in on code paths, but this new code will definitely benefit from fuzz testing. Based on my understanding of vulnerability discovery, this patch clearly hits the "WorthFuzzing=true" criteria: core subsystem modifications (I/O, memory management), refactoring (new data structures, new algorithms), and runtime assertions (defensive checks). The fact that it is now used by `afs`, and `cifs` makes this super-high priority for testing. Now, I'm digging deeper into how this will translate to the fuzzer configuration. I need to figure out: 1. `EnableConfigs`: What config options *must* be enabled to hit the code? `NETFS_SUPPORT` is obviously key. I'll need to verify if `NETFS_SUPPORT` can be directly set in a configuration or is only indirectly enabled, so I'm going to look at the Kconfig files. The fact that it's tristate with no prompt requires additional investigation. I'll need to figure out exactly how to enable it, especially if it's indirectly selected by the file systems. I need to make sure that the fuzzer has a known good starting configuration that enables this. `AFS_FS` and `CIFS` are also strong candidates. 2. `FocusSymbols`: Which of the new or modified functions are the most critical to target with fuzzing? I'll probably start |
| 10/2 | 2026/09/30 13:46 | tool | read-file |
0mArgs:{
"File": "fs/afs/Kconfig",
"FirstLine": 1,
"LineCount": 25
}Results: {
"Contents": " 1:\t# SPDX-License-Identifier: GPL-2.0-only\n 2:\tconfig AFS_FS\n 3:\t\ttristate \"Andrew File System support (AFS)\"\n 4:\t\tdepends on INET\n 5:\t\tselect AF_RXRPC\n 6:\t\tselect DNS_RESOLVER\n 7:\t\tselect NETFS_SUPPORT\n 8:\t\tselect CRYPTO_KRB5\n 9:\t\thelp\n 10:\t\t If you say Y here, you will get an experimental Andrew File System\n 11:\t\t driver. It currently only supports unsecured read-only AFS access.\n 12:\t\n 13:\t\t See \u003cfile:Documentation/filesystems/afs.rst\u003e for more information.\n 14:\t\n 15:\t\t If unsure, say N.\n 16:\t\n 17:\tconfig AFS_DEBUG\n 18:\t\tbool \"AFS dynamic debugging\"\n 19:\t\tdepends on AFS_FS\n 20:\t\thelp\n 21:\t\t Say Y here to make runtime controllable debugging messages appear.\n 22:\t\n 23:\t\t See \u003cfile:Documentation/filesystems/afs.rst\u003e for more information.\n 24:\t\n 25:\t\t If unsure, say N.\n"
} |
| 11/2 | 2026/09/30 13:46 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 69935 output: 35 thoughts: 15 Thoughts: Here's a summary of my thought process, as requested: **Diving into the Kernel Configuration for SMB Client** Okay, so I need to understand something related to the SMB client, likely a configuration option or dependency. My first instinct is to navigate the kernel source and start with the Kconfig file. It's the standard place to define and document build-time configuration options. So, let's take a look at `fs/smb/client/Kconfig`. That's where the heart of configuring the SMB client's features, and often any dependencies, will be defined. I'm expecting to find a list of options related to protocol versions, security mechanisms, caching behavior, and perhaps even features like support for specific SMB dialects. This Kconfig file should give me a solid overview of what's configurable, and I'll see where to go from there, for now I will check out its content. |
| 12/2 | 2026/09/30 13:47 | tool | read-file |
0mArgs:{
"File": "fs/smb/client/Kconfig",
"FirstLine": 1,
"LineCount": 35
}Results: {
"Contents": " 1:\t# SPDX-License-Identifier: GPL-2.0-only\n 2:\tconfig CIFS\n 3:\t\ttristate \"SMB3 and CIFS support (advanced network filesystem)\"\n 4:\t\tdepends on INET\n 5:\t\tselect NLS\n 6:\t\tselect NLS_UCS2_UTILS\n 7:\t\tselect CRYPTO\n 8:\t\tselect CRYPTO_AEAD2\n 9:\t\tselect CRYPTO_CCM\n 10:\t\tselect CRYPTO_GCM\n 11:\t\tselect CRYPTO_AES\n 12:\t\tselect CRYPTO_LIB_AES_CBC_MACS\n 13:\t\tselect CRYPTO_LIB_ARC4\n 14:\t\tselect CRYPTO_LIB_MD5\n 15:\t\tselect CRYPTO_LIB_SHA256\n 16:\t\tselect CRYPTO_LIB_SHA512\n 17:\t\tselect KEYS\n 18:\t\tselect DNS_RESOLVER\n 19:\t\tselect ASN1\n 20:\t\tselect OID_REGISTRY\n 21:\t\tselect NETFS_SUPPORT\n 22:\t\thelp\n 23:\t\t This is the client VFS module for the SMB3 family of network file\n 24:\t\t protocols (including the most recent, most secure dialect SMB3.1.1).\n 25:\t\t This module also includes support for earlier dialects such as\n 26:\t\t SMB2.1, SMB2 and even the old Common Internet File System (CIFS)\n 27:\t\t protocol. CIFS was the successor to the original network filesystem\n 28:\t\t protocol, Server Message Block (SMB ie SMB1), the native file sharing\n 29:\t\t mechanism for most early PC operating systems.\n 30:\t\n 31:\t\t The SMB3.1.1 protocol is supported by most modern operating systems\n 32:\t\t and NAS appliances (e.g. Samba, Windows 11, Windows Server 2022,\n 33:\t\t MacOS) and even in the cloud (e.g. Microsoft Azure) and also by the\n 34:\t\t Linux kernel server, ksmbd. Support for the older CIFS protocol was\n 35:\t\t included in Windows NT4, 2000 and XP (and later). Use of dialects\n"
} |
| 13/2 | 2026/09/30 13:47 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 70739 output: 28 thoughts: 50 Thoughts: Okay, here's my interpretation of those thoughts, tailored for an expert audience: **Relevant Filesystem Considerations** Alright, let's zero in. Thinking about network filesystems for this specific situation, `NETFS_SUPPORT`, `AFS_FS`, and `CIFS` are definitely jumping out as strong candidates for consideration. They're all pretty standard and have their own nuances in terms of performance and features, so understanding the specifics of each is crucial for making the right call. Hold on a second, though. Before I get too far down that path, let's not forget about `smbdirect`. It's worth digging into that, too. I'll need to check the code, and a quick glance at the kernel source suggests a likely area to start would be `fs/smb/smbdirect/connection.c`. I have to see how it's implemented and how it stacks up against the more established options. Gotta cover all the bases. |
| 14/2 | 2026/09/30 13:47 | tool | grepper |
0mArgs:{
"Expression": "smbdirect",
"PathPrefix": "fs/smb/"
}Results: {
"Output": "\nFull output is too long, showing 500 out of 3698 lines.\nUse more precise expression if possible.\n\nfs/smb/Kconfig=6=source \"fs/smb/server/Kconfig\"\nfs/smb/Kconfig:7:source \"fs/smb/smbdirect/Kconfig\"\nfs/smb/Kconfig-8-\n--\nfs/smb/Makefile=3=obj-$(CONFIG_SMBFS)\t\t+= common/\nfs/smb/Makefile:4:obj-$(CONFIG_SMBDIRECT)\t\t+= smbdirect/\nfs/smb/Makefile-5-obj-$(CONFIG_CIFS)\t\t+= client/\n--\nfs/smb/client/Kconfig=2=config CIFS\n--\nfs/smb/client/Kconfig-42-\t for DFS (hierarchical name space), secure per-user session\nfs/smb/client/Kconfig:43:\t establishment via Kerberos or NTLMv2, RDMA (smbdirect), advanced\nfs/smb/client/Kconfig-44-\t security features, per-share encryption, packet-signing, snapshots,\n--\nfs/smb/client/Makefile=29=cifs-$(CONFIG_CIFS_FSCACHE) += fscache.o\nfs/smb/client/Makefile-30-\nfs/smb/client/Makefile:31:cifs-$(CONFIG_CIFS_SMB_DIRECT) += smbdirect.o\nfs/smb/client/Makefile-32-\n--\nfs/smb/client/cifs_debug.c-24-#ifdef CONFIG_CIFS_SMB_DIRECT\nfs/smb/client/cifs_debug.c:25:#include \"smbdirect.h\"\nfs/smb/client/cifs_debug.c-26-#endif\n--\nfs/smb/client/cifsglob.h=1490=struct cifs_io_subrequest {\n--\nfs/smb/client/cifsglob.h-1505-#ifdef CONFIG_CIFS_SMB_DIRECT\nfs/smb/client/cifsglob.h:1506:\tstruct smbdirect_mr_io\t\t*mr;\nfs/smb/client/cifsglob.h-1507-#endif\n--\nfs/smb/client/cifsglob.h=1949=require use of the stronger protocol */\n--\nfs/smb/client/cifsglob.h-2057- *\t\t\t\t-\u003eoplock_break_cancelled\nfs/smb/client/cifsglob.h:2058: * smbdirect_mr-\u003emutex\t\tRDMA memory region management\t(SMBDirect only)\nfs/smb/client/cifsglob.h-2059- * mid_q_entry-\u003emid_lock\tmid_q_entry-\u003ecallback alloc_mid\n--\nfs/smb/client/cifssmb.c-37-#include \"fscache.h\"\nfs/smb/client/cifssmb.c:38:#include \"smbdirect.h\"\nfs/smb/client/cifssmb.c-39-#ifdef CONFIG_CIFS_DFS_UPCALL\n--\nfs/smb/client/connect.c-44-#include \"smb2proto.h\"\nfs/smb/client/connect.c:45:#include \"smbdirect.h\"\nfs/smb/client/connect.c-46-#include \"dns_resolve.h\"\n--\nfs/smb/client/file.c-33-#include \"fscache.h\"\nfs/smb/client/file.c:34:#include \"smbdirect.h\"\nfs/smb/client/file.c-35-#include \"fs_context.h\"\n--\nfs/smb/client/file.c=46=static void cifs_prepare_write(struct netfs_io_subrequest *subreq)\n--\nfs/smb/client/file.c-99-\tif (server-\u003esmbd_conn) {\nfs/smb/client/file.c:100:\t\tconst struct smbdirect_socket_parameters *sp =\nfs/smb/client/file.c-101-\t\t\tsmbd_get_parameters(server-\u003esmbd_conn);\n--\nfs/smb/client/file.c=153=static int cifs_prepare_read(struct netfs_io_subrequest *subreq)\n--\nfs/smb/client/file.c-193-\tif (server-\u003esmbd_conn) {\nfs/smb/client/file.c:194:\t\tconst struct smbdirect_socket_parameters *sp =\nfs/smb/client/file.c-195-\t\t\tsmbd_get_parameters(server-\u003esmbd_conn);\n--\nfs/smb/client/smb1transport.c-29-#include \"cifs_debug.h\"\nfs/smb/client/smb1transport.c:30:#include \"smbdirect.h\"\nfs/smb/client/smb1transport.c-31-#include \"compress.h\"\n--\nfs/smb/client/smb2ops.c-27-#include \"cifs_ioctl.h\"\nfs/smb/client/smb2ops.c:28:#include \"smbdirect.h\"\nfs/smb/client/smb2ops.c-29-#include \"fscache.h\"\n--\nfs/smb/client/smb2ops.c=508=smb3_negotiate_wsize(struct cifs_tcon *tcon, struct smb3_fs_context *ctx)\n--\nfs/smb/client/smb2ops.c-517-\tif (server-\u003erdma) {\nfs/smb/client/smb2ops.c:518:\t\tconst struct smbdirect_socket_parameters *sp =\nfs/smb/client/smb2ops.c-519-\t\t\tsmbd_get_parameters(server-\u003esmbd_conn);\n--\nfs/smb/client/smb2ops.c=559=smb3_negotiate_rsize(struct cifs_tcon *tcon, struct smb3_fs_context *ctx)\n--\nfs/smb/client/smb2ops.c-568-\tif (server-\u003erdma) {\nfs/smb/client/smb2ops.c:569:\t\tconst struct smbdirect_socket_parameters *sp =\nfs/smb/client/smb2ops.c-570-\t\t\tsmbd_get_parameters(server-\u003esmbd_conn);\n--\nfs/smb/client/smb2pdu.c-38-#include \"cifs_spnego.h\"\nfs/smb/client/smb2pdu.c:39:#include \"smbdirect.h\"\nfs/smb/client/smb2pdu.c-40-#include \"trace.h\"\n--\nfs/smb/client/smb2pdu.c=4585=smb2_new_read_req(void **buf, unsigned int *total_len,\n--\nfs/smb/client/smb2pdu.c-4624-\t * If we want to do a RDMA write, fill in and append\nfs/smb/client/smb2pdu.c:4625:\t * smbdirect_buffer_descriptor_v1 to the end of read request\nfs/smb/client/smb2pdu.c-4626-\t */\nfs/smb/client/smb2pdu.c-4627-\tif (rdata \u0026\u0026 smb3_use_rdma_offload(io_parms)) {\nfs/smb/client/smb2pdu.c:4628:\t\tstruct smbdirect_buffer_descriptor_v1 *v1;\nfs/smb/client/smb2pdu.c-4629-\t\tbool need_invalidate = server-\u003edialect == SMB30_PROT_ID;\n--\nfs/smb/client/smb2pdu.c-4643-\t\treq-\u003eReadChannelInfoLength =\nfs/smb/client/smb2pdu.c:4644:\t\t\tcpu_to_le16(sizeof(struct smbdirect_buffer_descriptor_v1));\nfs/smb/client/smb2pdu.c:4645:\t\tv1 = (struct smbdirect_buffer_descriptor_v1 *) \u0026req-\u003eBuffer[0];\nfs/smb/client/smb2pdu.c-4646-\t\tsmbd_mr_fill_buffer_descriptor(rdata-\u003emr, v1);\n--\nfs/smb/client/smb2pdu.c=5157=smb2_async_writev(struct cifs_io_subrequest *wdata)\n--\nfs/smb/client/smb2pdu.c-5226-\t * If we want to do a server RDMA read, fill in and append\nfs/smb/client/smb2pdu.c:5227:\t * smbdirect_buffer_descriptor_v1 to the end of write request\nfs/smb/client/smb2pdu.c-5228-\t */\nfs/smb/client/smb2pdu.c-5229-\tif (smb3_use_rdma_offload(io_parms)) {\nfs/smb/client/smb2pdu.c:5230:\t\tstruct smbdirect_buffer_descriptor_v1 *v1;\nfs/smb/client/smb2pdu.c-5231-\t\tbool need_invalidate = server-\u003edialect == SMB30_PROT_ID;\n--\nfs/smb/client/smb2pdu.c-5248-\t\treq-\u003eWriteChannelInfoLength =\nfs/smb/client/smb2pdu.c:5249:\t\t\tcpu_to_le16(sizeof(struct smbdirect_buffer_descriptor_v1));\nfs/smb/client/smb2pdu.c:5250:\t\tv1 = (struct smbdirect_buffer_descriptor_v1 *) \u0026req-\u003eBuffer[0];\nfs/smb/client/smb2pdu.c-5251-\t\tsmbd_mr_fill_buffer_descriptor(wdata-\u003emr, v1);\n--\nfs/smb/client/smbdirect.c-7-\nfs/smb/client/smbdirect.c:8:#include \"smbdirect.h\"\nfs/smb/client/smbdirect.c-9-#include \"cifs_debug.h\"\n--\nfs/smb/client/smbdirect.c=104=MODULE_PARM_DESC(smbd_logging_level,\n--\nfs/smb/client/smbdirect.c-106-\nfs/smb/client/smbdirect.c:107:static bool smbd_logging_needed(struct smbdirect_socket *sc,\nfs/smb/client/smbdirect.c-108-\t\t\t\tvoid *private_ptr,\n--\nfs/smb/client/smbdirect.c-132-\nfs/smb/client/smbdirect.c:133:static void smbd_logging_vaprintf(struct smbdirect_socket *sc,\nfs/smb/client/smbdirect.c-134-\t\t\t\t const char *func,\n--\nfs/smb/client/smbdirect.c=145=do {\t\t\t\t\t\t\t\t\t\\\n--\nfs/smb/client/smbdirect.c-166-\nfs/smb/client/smbdirect.c:167:static int smbd_post_send_full_iter(struct smbdirect_socket *sc,\nfs/smb/client/smbdirect.c:168:\t\t\t\t struct smbdirect_send_batch *batch,\nfs/smb/client/smbdirect.c-169-\t\t\t\t struct iov_iter *iter,\n--\nfs/smb/client/smbdirect.c-174-\t/*\nfs/smb/client/smbdirect.c:175:\t * smbdirect_connection_send_single_iter() respects the\nfs/smb/client/smbdirect.c-176-\t * negotiated max_send_size, so we need to\n--\nfs/smb/client/smbdirect.c-182-\nfs/smb/client/smbdirect.c:183:\t\trc = smbdirect_connection_send_single_iter(sc,\nfs/smb/client/smbdirect.c-184-\t\t\t\t\t\t\t batch,\n--\nfs/smb/client/smbdirect.c=202=void smbd_destroy(struct TCP_Server_Info *server)\n--\nfs/smb/client/smbdirect.c-210-\nfs/smb/client/smbdirect.c:211:\tsmbdirect_socket_release(info-\u003esocket);\nfs/smb/client/smbdirect.c-212-\n--\nfs/smb/client/smbdirect.c=252=static struct smbd_connection *_smbd_get_connection(\n--\nfs/smb/client/smbdirect.c-256-\tstruct smbd_connection *info;\nfs/smb/client/smbdirect.c:257:\tstruct smbdirect_socket *sc;\nfs/smb/client/smbdirect.c:258:\tstruct smbdirect_socket_parameters init_params = {};\nfs/smb/client/smbdirect.c:259:\tstruct smbdirect_socket_parameters *sp;\nfs/smb/client/smbdirect.c-260-\t__be16 *sport;\n--\nfs/smb/client/smbdirect.c-305-\t\treturn NULL;\nfs/smb/client/smbdirect.c:306:\tret = smbdirect_socket_create_kern(net, \u0026sc);\nfs/smb/client/smbdirect.c-307-\tif (ret)\nfs/smb/client/smbdirect.c-308-\t\tgoto socket_init_failed;\nfs/smb/client/smbdirect.c:309:\tsmbdirect_socket_set_logging(sc, NULL, smbd_logging_needed, smbd_logging_vaprintf);\nfs/smb/client/smbdirect.c:310:\tret = smbdirect_socket_set_initial_parameters(sc, sp);\nfs/smb/client/smbdirect.c-311-\tif (ret)\nfs/smb/client/smbdirect.c-312-\t\tgoto set_params_failed;\nfs/smb/client/smbdirect.c:313:\tret = smbdirect_socket_set_kernel_settings(sc, IB_POLL_SOFTIRQ, GFP_KERNEL);\nfs/smb/client/smbdirect.c-314-\tif (ret)\n--\nfs/smb/client/smbdirect.c-323-\nfs/smb/client/smbdirect.c:324:\tret = smbdirect_connect_sync(sc, dstaddr);\nfs/smb/client/smbdirect.c-325-\tif (ret) {\n--\nfs/smb/client/smbdirect.c-336-set_params_failed:\nfs/smb/client/smbdirect.c:337:\tsmbdirect_socket_release(sc);\nfs/smb/client/smbdirect.c-338-socket_init_failed:\n--\nfs/smb/client/smbdirect.c-342-\nfs/smb/client/smbdirect.c:343:const struct smbdirect_socket_parameters *smbd_get_parameters(struct smbd_connection *conn)\nfs/smb/client/smbdirect.c-344-{\nfs/smb/client/smbdirect.c-345-\tif (unlikely(!conn-\u003esocket)) {\nfs/smb/client/smbdirect.c:346:\t\tstatic const struct smbdirect_socket_parameters zero_params;\nfs/smb/client/smbdirect.c-347-\n--\nfs/smb/client/smbdirect.c-350-\nfs/smb/client/smbdirect.c:351:\treturn smbdirect_socket_get_current_parameters(conn-\u003esocket);\nfs/smb/client/smbdirect.c-352-}\n--\nfs/smb/client/smbdirect.c=354=struct smbd_connection *smbd_get_connection(\n--\nfs/smb/client/smbdirect.c-357-\tstruct smbd_connection *ret;\nfs/smb/client/smbdirect.c:358:\tconst struct smbdirect_socket_parameters *sp;\nfs/smb/client/smbdirect.c-359-\tint port = SMBD_PORT;\n--\nfs/smb/client/smbdirect.c=396=int smbd_recv(struct smbd_connection *info, struct msghdr *msg)\nfs/smb/client/smbdirect.c-397-{\nfs/smb/client/smbdirect.c:398:\tstruct smbdirect_socket *sc = info-\u003esocket;\nfs/smb/client/smbdirect.c-399-\nfs/smb/client/smbdirect.c:400:\tif (!smbdirect_connection_is_connected(sc))\nfs/smb/client/smbdirect.c-401-\t\treturn -ENOTCONN;\nfs/smb/client/smbdirect.c-402-\nfs/smb/client/smbdirect.c:403:\treturn smbdirect_connection_recvmsg(sc, msg, 0);\nfs/smb/client/smbdirect.c-404-}\n--\nfs/smb/client/smbdirect.c=412=int smbd_send(struct TCP_Server_Info *server,\n--\nfs/smb/client/smbdirect.c-415-\tstruct smbd_connection *info = server-\u003esmbd_conn;\nfs/smb/client/smbdirect.c:416:\tstruct smbdirect_socket *sc = info-\u003esocket;\nfs/smb/client/smbdirect.c:417:\tconst struct smbdirect_socket_parameters *sp = smbd_get_parameters(info);\nfs/smb/client/smbdirect.c-418-\tstruct smb_rqst *rqst;\nfs/smb/client/smbdirect.c-419-\tstruct iov_iter iter;\nfs/smb/client/smbdirect.c:420:\tstruct smbdirect_send_batch_storage bstorage;\nfs/smb/client/smbdirect.c:421:\tstruct smbdirect_send_batch *batch;\nfs/smb/client/smbdirect.c-422-\tunsigned int remaining_data_length, klen;\n--\nfs/smb/client/smbdirect.c-425-\nfs/smb/client/smbdirect.c:426:\tif (!smbdirect_connection_is_connected(sc))\nfs/smb/client/smbdirect.c-427-\t\treturn -EAGAIN;\n--\nfs/smb/client/smbdirect.c-448-\trqst_idx = 0;\nfs/smb/client/smbdirect.c:449:\tbatch = smbdirect_init_send_batch_storage(\u0026bstorage, false, 0);\nfs/smb/client/smbdirect.c-450-\tdo {\n--\nfs/smb/client/smbdirect.c-487-\nfs/smb/client/smbdirect.c:488:\trc = smbdirect_connection_send_batch_flush(sc, batch, true);\nfs/smb/client/smbdirect.c-489-\tif (unlikely(!rc \u0026\u0026 error))\n--\nfs/smb/client/smbdirect.c-499-\terror = rc;\nfs/smb/client/smbdirect.c:500:\trc = smbdirect_connection_send_wait_zero_pending(sc);\nfs/smb/client/smbdirect.c-501-\tif (unlikely(rc \u0026\u0026 !error))\n--\nfs/smb/client/smbdirect.c-516- */\nfs/smb/client/smbdirect.c:517:struct smbdirect_mr_io *smbd_register_mr(struct smbd_connection *info,\nfs/smb/client/smbdirect.c-518-\t\t\t\t struct iov_iter *iter,\n--\nfs/smb/client/smbdirect.c-520-{\nfs/smb/client/smbdirect.c:521:\tstruct smbdirect_socket *sc = info-\u003esocket;\nfs/smb/client/smbdirect.c-522-\nfs/smb/client/smbdirect.c:523:\tif (!smbdirect_connection_is_connected(sc))\nfs/smb/client/smbdirect.c-524-\t\treturn NULL;\nfs/smb/client/smbdirect.c-525-\nfs/smb/client/smbdirect.c:526:\treturn smbdirect_connection_register_mr_io(sc, iter, writing, need_invalidate);\nfs/smb/client/smbdirect.c-527-}\nfs/smb/client/smbdirect.c-528-\nfs/smb/client/smbdirect.c:529:void smbd_mr_fill_buffer_descriptor(struct smbdirect_mr_io *mr,\nfs/smb/client/smbdirect.c:530:\t\t\t\t struct smbdirect_buffer_descriptor_v1 *v1)\nfs/smb/client/smbdirect.c-531-{\nfs/smb/client/smbdirect.c:532:\tsmbdirect_mr_io_fill_buffer_descriptor(mr, v1);\nfs/smb/client/smbdirect.c-533-}\n--\nfs/smb/client/smbdirect.c-540- */\nfs/smb/client/smbdirect.c:541:void smbd_deregister_mr(struct smbdirect_mr_io *mr)\nfs/smb/client/smbdirect.c-542-{\nfs/smb/client/smbdirect.c:543:\tsmbdirect_connection_deregister_mr_io(mr);\nfs/smb/client/smbdirect.c-544-}\n--\nfs/smb/client/smbdirect.c=546=void smbd_debug_proc_show(struct TCP_Server_Info *server, struct seq_file *m)\n--\nfs/smb/client/smbdirect.c-555-\nfs/smb/client/smbdirect.c:556:\tsmbdirect_connection_legacy_debug_proc_show(server-\u003esmbd_conn-\u003esocket,\nfs/smb/client/smbdirect.c-557-\t\t\t\t\t\t server-\u003erdma_readwrite_threshold,\n--\nfs/smb/client/smbdirect.h-14-\nfs/smb/client/smbdirect.h:15:#include \u003clinux/smbdirect.h\u003e\nfs/smb/client/smbdirect.h-16-\n--\nfs/smb/client/smbdirect.h=26=struct smbd_connection {\nfs/smb/client/smbdirect.h:27:\tstruct smbdirect_socket *socket;\nfs/smb/client/smbdirect.h-28-};\n--\nfs/smb/client/smbdirect.h=31=struct smbd_connection *smbd_get_connection(\n--\nfs/smb/client/smbdirect.h-33-\nfs/smb/client/smbdirect.h:34:const struct smbdirect_socket_parameters *smbd_get_parameters(struct smbd_connection *conn);\nfs/smb/client/smbdirect.h-35-\n--\nfs/smb/client/smbdirect.h=43=int smbd_send(struct TCP_Server_Info *server,\n--\nfs/smb/client/smbdirect.h-46-/* Interfaces to register and deregister MR for RDMA read/write */\nfs/smb/client/smbdirect.h:47:struct smbdirect_mr_io *smbd_register_mr(\nfs/smb/client/smbdirect.h-48-\tstruct smbd_connection *info, struct iov_iter *iter,\nfs/smb/client/smbdirect.h-49-\tbool writing, bool need_invalidate);\nfs/smb/client/smbdirect.h:50:void smbd_mr_fill_buffer_descriptor(struct smbdirect_mr_io *mr,\nfs/smb/client/smbdirect.h:51:\t\t\t\t struct smbdirect_buffer_descriptor_v1 *v1);\nfs/smb/client/smbdirect.h:52:void smbd_deregister_mr(struct smbdirect_mr_io *mr);\nfs/smb/client/smbdirect.h-53-\n--\nfs/smb/client/transport.c-29-#include \"smb2proto.h\"\nfs/smb/client/transport.c:30:#include \"smbdirect.h\"\nfs/smb/client/transport.c-31-#include \"compress.h\"\n--\nfs/smb/server/Kconfig=1=config SMB_SERVER\n--\nfs/smb/server/Kconfig-40-\t Secure negotiate, Pre-authentication integrity, oplock/lease,\nfs/smb/server/Kconfig:41:\t compound requests, multi-credit, packet signing, RDMA(smbdirect),\nfs/smb/server/Kconfig-42-\t smb3 encryption, copy-offload, secure per-user session\n--\nfs/smb/server/connection.c=56=static const char *ksmbd_conn_transport_string(struct ksmbd_conn *conn)\n--\nfs/smb/server/connection.c-58-\tif (conn-\u003etransport-\u003eops-\u003erdma_read || conn-\u003etransport-\u003eops-\u003erdma_write)\nfs/smb/server/connection.c:59:\t\treturn \"smbdirect\";\nfs/smb/server/connection.c-60-\treturn \"tcp\";\n--\nfs/smb/server/connection.c=565=int ksmbd_conn_rdma_read(struct ksmbd_conn *conn,\nfs/smb/server/connection.c-566-\t\t\t void *buf, unsigned int buflen,\nfs/smb/server/connection.c:567:\t\t\t struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/connection.c-568-\t\t\t unsigned int desc_len)\n--\nfs/smb/server/connection.c=579=int ksmbd_conn_rdma_write(struct ksmbd_conn *conn,\nfs/smb/server/connection.c-580-\t\t\t void *buf, unsigned int buflen,\nfs/smb/server/connection.c:581:\t\t\t struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/connection.c-582-\t\t\t unsigned int desc_len)\n--\nfs/smb/server/connection.h-24-\nfs/smb/server/connection.h:25:struct smbdirect_buffer_descriptor_v1;\nfs/smb/server/connection.h-26-struct ksmbd_session;\n--\nfs/smb/server/connection.h=171=struct ksmbd_transport_ops {\n--\nfs/smb/server/connection.h-179-\t\t\t void *buf, unsigned int len,\nfs/smb/server/connection.h:180:\t\t\t struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/connection.h-181-\t\t\t unsigned int desc_len);\n--\nfs/smb/server/connection.h-183-\t\t\t void *buf, unsigned int len,\nfs/smb/server/connection.h:184:\t\t\t struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/connection.h-185-\t\t\t unsigned int desc_len);\n--\nfs/smb/server/connection.h=218=int ksmbd_conn_rdma_read(struct ksmbd_conn *conn,\nfs/smb/server/connection.h-219-\t\t\t void *buf, unsigned int buflen,\nfs/smb/server/connection.h:220:\t\t\t struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/connection.h-221-\t\t\t unsigned int desc_len);\nfs/smb/server/connection.h=222=int ksmbd_conn_rdma_write(struct ksmbd_conn *conn,\nfs/smb/server/connection.h-223-\t\t\t void *buf, unsigned int buflen,\nfs/smb/server/connection.h:224:\t\t\t struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/connection.h-225-\t\t\t unsigned int desc_len);\n--\nfs/smb/server/smb2pdu.c=8729=static int smb2_set_rdma_key(struct ksmbd_work *work,\nfs/smb/server/smb2pdu.c:8730:\t\t\t struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/smb2pdu.c-8731-\t\t\t __le32 Channel, __le16 channel_info_len)\n--\nfs/smb/server/smb2pdu.c=8833=struct smb2_rdma_write_transform {\nfs/smb/server/smb2pdu.c:8834:\tstruct smbdirect_buffer_descriptor_v1 *desc;\nfs/smb/server/smb2pdu.c-8835-\tstruct smb2_rdma_crypto_transform *crypto;\n--\nfs/smb/server/smb2pdu.c=8851=static unsigned int smb2_current_req_len(struct ksmbd_work *work,\n--\nfs/smb/server/smb2pdu.c-8867- */\nfs/smb/server/smb2pdu.c:8868:static int check_rdma_desc(struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/smb2pdu.c-8869-\t\t\t unsigned int desc_len,\n--\nfs/smb/server/smb2pdu.c=8896=static int smb2_parse_rdma_write_transform(struct ksmbd_work *work,\n--\nfs/smb/server/smb2pdu.c-8964-\nfs/smb/server/smb2pdu.c:8965:\tinfo-\u003edesc = (struct smbdirect_buffer_descriptor_v1 *)\nfs/smb/server/smb2pdu.c-8966-\t\t((char *)transform + desc_offset);\n--\nfs/smb/server/smb2pdu.c=8994=static ssize_t smb2_read_rdma(struct ksmbd_work *work,\n--\nfs/smb/server/smb2pdu.c-9000-\terr = ksmbd_conn_rdma_write(work-\u003econn, data_buf, length,\nfs/smb/server/smb2pdu.c:9001:\t\t\t\t (struct smbdirect_buffer_descriptor_v1 *)\nfs/smb/server/smb2pdu.c-9002-\t\t\t\t ((char *)req + le16_to_cpu(req-\u003eReadChannelInfoOffset)),\n--\nfs/smb/server/smb2pdu.c=9016=int smb2_read(struct ksmbd_work *work)\n--\nfs/smb/server/smb2pdu.c-9080-\t\tunsigned int req_len = smb2_current_req_len(work, \u0026req-\u003ehdr);\nfs/smb/server/smb2pdu.c:9081:\t\tstruct smbdirect_buffer_descriptor_v1 *desc;\nfs/smb/server/smb2pdu.c-9082-\n--\nfs/smb/server/smb2pdu.c-9088-\t\t}\nfs/smb/server/smb2pdu.c:9089:\t\tdesc = (struct smbdirect_buffer_descriptor_v1 *)\nfs/smb/server/smb2pdu.c-9090-\t\t\t((char *)req + ch_offset);\n--\nfs/smb/server/smb2pdu.c=9347=static ssize_t smb2_write_rdma(struct ksmbd_work *work,\nfs/smb/server/smb2pdu.c:9348:\t\t\t struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/smb2pdu.c-9349-\t\t\t unsigned int desc_len,\n--\nfs/smb/server/smb2pdu.c=9408=int smb2_write(struct ksmbd_work *work)\n--\nfs/smb/server/smb2pdu.c-9413-\tstruct smb2_rdma_write_transform *rdma_info = NULL;\nfs/smb/server/smb2pdu.c:9414:\tstruct smbdirect_buffer_descriptor_v1 *rdma_desc = NULL;\nfs/smb/server/smb2pdu.c-9415-\tunsigned int rdma_desc_len = 0;\n--\nfs/smb/server/smb2pdu.c-9522-\t\t} else {\nfs/smb/server/smb2pdu.c:9523:\t\t\trdma_desc = (struct smbdirect_buffer_descriptor_v1 *)\nfs/smb/server/smb2pdu.c-9524-\t\t\t\t((char *)req + ch_offset);\n--\nfs/smb/server/transport_rdma.c=81=static struct smb_direct_listener {\n--\nfs/smb/server/transport_rdma.c-85-\nfs/smb/server/transport_rdma.c:86:\tstruct smbdirect_socket *socket;\nfs/smb/server/transport_rdma.c-87-} smb_direct_ib_listener, smb_direct_iw_listener;\n--\nfs/smb/server/transport_rdma.c=89=struct smb_direct_transport {\n--\nfs/smb/server/transport_rdma.c-91-\nfs/smb/server/transport_rdma.c:92:\tstruct smbdirect_socket *socket;\nfs/smb/server/transport_rdma.c-93-};\nfs/smb/server/transport_rdma.c-94-\nfs/smb/server/transport_rdma.c:95:static bool smb_direct_logging_needed(struct smbdirect_socket *sc,\nfs/smb/server/transport_rdma.c-96-\t\t\t\t void *private_ptr,\n--\nfs/smb/server/transport_rdma.c-139-\nfs/smb/server/transport_rdma.c:140:static void smb_direct_logging_vaprintf(struct smbdirect_socket *sc,\nfs/smb/server/transport_rdma.c-141-\t\t\t\t\tconst char *func,\n--\nfs/smb/server/transport_rdma.c=166=unsigned int get_smbd_max_read_write_size(struct ksmbd_transport *kt)\n--\nfs/smb/server/transport_rdma.c-168-\tstruct smb_direct_transport *t;\nfs/smb/server/transport_rdma.c:169:\tconst struct smbdirect_socket_parameters *sp;\nfs/smb/server/transport_rdma.c-170-\n--\nfs/smb/server/transport_rdma.c-174-\tt = SMBD_TRANS(kt);\nfs/smb/server/transport_rdma.c:175:\tsp = smbdirect_socket_get_current_parameters(t-\u003esocket);\nfs/smb/server/transport_rdma.c-176-\n--\nfs/smb/server/transport_rdma.c-179-\nfs/smb/server/transport_rdma.c:180:static struct smb_direct_transport *alloc_transport(struct smbdirect_socket *sc)\nfs/smb/server/transport_rdma.c-181-{\n--\nfs/smb/server/transport_rdma.c=209=static void smb_direct_free_transport(struct ksmbd_transport *kt)\n--\nfs/smb/server/transport_rdma.c-212-\nfs/smb/server/transport_rdma.c:213:\tsmbdirect_socket_release(t-\u003esocket);\nfs/smb/server/transport_rdma.c-214-\tkfree(t);\n--\nfs/smb/server/transport_rdma.c=217=static void free_transport(struct smb_direct_transport *t)\nfs/smb/server/transport_rdma.c-218-{\nfs/smb/server/transport_rdma.c:219:\tsmbdirect_socket_shutdown(t-\u003esocket);\nfs/smb/server/transport_rdma.c-220-\tksmbd_conn_free(KSMBD_TRANS(t)-\u003econn);\n--\nfs/smb/server/transport_rdma.c=223=static int smb_direct_read(struct ksmbd_transport *t, char *buf,\n--\nfs/smb/server/transport_rdma.c-226-\tstruct smb_direct_transport *st = SMBD_TRANS(t);\nfs/smb/server/transport_rdma.c:227:\tstruct smbdirect_socket *sc = st-\u003esocket;\nfs/smb/server/transport_rdma.c-228-\tstruct msghdr msg = { .msg_flags = 0, };\n--\nfs/smb/server/transport_rdma.c-236-\nfs/smb/server/transport_rdma.c:237:\tret = smbdirect_connection_recvmsg(sc, \u0026msg, 0);\nfs/smb/server/transport_rdma.c-238-\tif (ret == -ERESTARTSYS)\n--\nfs/smb/server/transport_rdma.c=243=static int smb_direct_writev(struct ksmbd_transport *t,\n--\nfs/smb/server/transport_rdma.c-246-\tstruct smb_direct_transport *st = SMBD_TRANS(t);\nfs/smb/server/transport_rdma.c:247:\tstruct smbdirect_socket *sc = st-\u003esocket;\nfs/smb/server/transport_rdma.c-248-\tstruct iov_iter iter;\n--\nfs/smb/server/transport_rdma.c-252-\nfs/smb/server/transport_rdma.c:253:\treturn smbdirect_connection_send_iter(sc, \u0026iter, 0,\nfs/smb/server/transport_rdma.c-254-\t\t\t\t\t tx-\u003eneed_invalidate_rkey,\n--\nfs/smb/server/transport_rdma.c=258=static int smb_direct_rdma_write(struct ksmbd_transport *t,\nfs/smb/server/transport_rdma.c-259-\t\t\t\t void *buf, unsigned int buflen,\nfs/smb/server/transport_rdma.c:260:\t\t\t\t struct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/transport_rdma.c-261-\t\t\t\t unsigned int desc_len)\n--\nfs/smb/server/transport_rdma.c-263-\tstruct smb_direct_transport *st = SMBD_TRANS(t);\nfs/smb/server/transport_rdma.c:264:\tstruct smbdirect_socket *sc = st-\u003esocket;\nfs/smb/server/transport_rdma.c-265-\nfs/smb/server/transport_rdma.c:266:\treturn smbdirect_connection_rdma_xmit(sc, buf, buflen,\nfs/smb/server/transport_rdma.c-267-\t\t\t\t\t desc, desc_len, false);\n--\nfs/smb/server/transport_rdma.c=270=static int smb_direct_rdma_read(struct ksmbd_transport *t,\nfs/smb/server/transport_rdma.c-271-\t\t\t\tvoid *buf, unsigned int buflen,\nfs/smb/server/transport_rdma.c:272:\t\t\t\tstruct smbdirect_buffer_descriptor_v1 *desc,\nfs/smb/server/transport_rdma.c-273-\t\t\t\tunsigned int desc_len)\n--\nfs/smb/server/transport_rdma.c-275-\tstruct smb_direct_transport *st = SMBD_TRANS(t);\nfs/smb/server/transport_rdma.c:276:\tstruct smbdirect_socket *sc = st-\u003esocket;\nfs/smb/server/transport_rdma.c-277-\nfs/smb/server/transport_rdma.c:278:\treturn smbdirect_connection_rdma_xmit(sc, buf, buflen,\nfs/smb/server/transport_rdma.c-279-\t\t\t\t\t desc, desc_len, true);\n--\nfs/smb/server/transport_rdma.c=282=static void smb_direct_disconnect(struct ksmbd_transport *t)\n--\nfs/smb/server/transport_rdma.c-284-\tstruct smb_direct_transport *st = SMBD_TRANS(t);\nfs/smb/server/transport_rdma.c:285:\tstruct smbdirect_socket *sc = st-\u003esocket;\nfs/smb/server/transport_rdma.c-286-\n--\nfs/smb/server/transport_rdma.c=292=static void smb_direct_shutdown(struct ksmbd_transport *t)\n--\nfs/smb/server/transport_rdma.c-294-\tstruct smb_direct_transport *st = SMBD_TRANS(t);\nfs/smb/server/transport_rdma.c:295:\tstruct smbdirect_socket *sc = st-\u003esocket;\nfs/smb/server/transport_rdma.c-296-\n--\nfs/smb/server/transport_rdma.c-298-\nfs/smb/server/transport_rdma.c:299:\tsmbdirect_socket_shutdown(sc);\nfs/smb/server/transport_rdma.c-300-}\n--\nfs/smb/server/transport_rdma.c=302=static int smb_direct_new_connection(struct smb_direct_listener *listener,\nfs/smb/server/transport_rdma.c:303:\t\t\t\t struct smbdirect_socket *client_sc)\nfs/smb/server/transport_rdma.c-304-{\n--\nfs/smb/server/transport_rdma.c-310-\tif (!t) {\nfs/smb/server/transport_rdma.c:311:\t\tsmbdirect_socket_release(client_sc);\nfs/smb/server/transport_rdma.c-312-\t\treturn -ENOMEM;\n--\nfs/smb/server/transport_rdma.c=330=static int smb_direct_listener_kthread_fn(void *p)\n--\nfs/smb/server/transport_rdma.c-332-\tstruct smb_direct_listener *listener = (struct smb_direct_listener *)p;\nfs/smb/server/transport_rdma.c:333:\tstruct smbdirect_socket *client_sc = NULL;\nfs/smb/server/transport_rdma.c-334-\n--\nfs/smb/server/transport_rdma.c-340-\t\t\tbreak;\nfs/smb/server/transport_rdma.c:341:\t\tclient_sc = smbdirect_socket_accept(listener-\u003esocket, timeo, \u0026arg);\n\n"
} |
| 15/2 | 2026/09/30 13:47 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 82438 output: 37 thoughts: 44 Thoughts: Okay, here's my thought process on this, from my perspective, tailored for a fellow expert: **Investigating SMBDirect Configuration in the Kernel** Alright, let's dive into this. I've got a hunch that something related to SMBDirect is configured within the kernel, so I'm starting with a quick file system search. My initial point of interest is the `fs/smb/smbdirect/connection.c` file. The presence of `CONFIG_SMBDIRECT` in `connection.c` is a pretty good indicator that SMBDirect itself is a compile-time option. Now, I need to confirm how that option is *actually* set. Where's the configuration for `CONFIG_SMBDIRECT` defined? That means I need to jump over and look at the `fs/smb/smbdirect/Kconfig` file. That's the logical place where this particular configuration option would be defined and exposed for the build system. From there I can determine the various conditions, dependencies, and default states that govern whether SMBDirect gets compiled in or not. That will give me the full picture. |
| 16/2 | 2026/09/30 13:47 | tool | read-file |
0mArgs:{
"File": "fs/smb/smbdirect/Kconfig",
"FirstLine": 1,
"LineCount": 20
}Results: {
"Contents": " 1:\t# SPDX-License-Identifier: GPL-2.0-or-later\n 2:\t#\n 3:\t# smbdirect configuration\n 4:\t\n 5:\tconfig SMBDIRECT\n 6:\t\tdef_tristate n\n 7:\t\tdepends on INFINIBAND \u0026\u0026 INFINIBAND_ADDR_TRANS\n 8:\t\tdepends on m || INFINIBAND=y\n 9:\t\tselect SG_POOL\n"
} |
| 17/2 | 2026/09/30 13:47 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 82872 output: 34 thoughts: 1528 Thoughts: **Analyzing Kernel Code for Fuzzing Focus** Okay, so I'm diving into this patch with the goal of identifying key functions for fuzzing with syzkaller. First, I need to understand the configuration implications: `SMBDIRECT` is for RDMA SMB, that's clear. Now, what about `NETFS_SUPPORT`? It seems it's linked to `AFS_FS`, `CIFS`, or `9P_FS`, but can also be enabled directly, good to know. The `TEST_IOV_ITER` config is just a KUnit test; not relevant for our fuzzing efforts. Now, for the crux of it: what *new* functionality does this patch introduce that is not ubiquitous and thus is worth focusing fuzzing attention on? I need to list the key functions. Let me review them again: 1. `bvecq_alloc_one` 2. `bvecq_alloc_chain` 3. `bvecq_alloc_buffer2` 4. `bvecq_put` 5. `bvecq_expand_buffer` 6. `iov_iter_bvec_queue` 7. `iov_iter_bvecq_advance` 8. `iov_iter_bvecq_revert` 9. `iter_bvecq_get_pages` 10. `iov_iter_extract_bvecq_pages` 11. `extract_bvecq_to_sg` 12. `rolling_buffer_init` 13. `rolling_buffer_make_space` 14. `rolling_buffer_bulk_load_from_ra` 15. `rolling_buffer_append` 16. `rolling_buffer_delete_spent` 17. `netfs_writeback_single` 18. `cifs_copy_iter_to_bvecq` 19. `cifs_copy_bvecq_to_iter` 20. `afs_dir_get_block` 21. `afs_dir_find_block` Now, let's analyze those. I need to be precise on static vs. exported symbols, that's important for syzkaller. Let me verify the static status of some of those iov_iter functions: `iov_iter_bvecq_advance` and `iov_iter_bvecq_revert`. They're *static* in `lib/iov_iter.c`, as I suspected. Also, `iter_count_bvecq_pages`, `iter_bvecq_get_pages`, `iov_iter_extract_bvecq_pages`, and `iov_iter_alignment_bvecq`. Those are all static too. Aha, but `iov_iter_bvec_queue` is *exported* via `EXPORT_SYMBOL`. Good, those will be easier to focus on. `iov_iter_advance` and `iov_iter_revert` call the bvecq-specific functions, but they're ubiquitous, so not as interesting. Now, let's drill down into each file where these functions reside, to clarify which of the listed symbols are exported or static: * `fs/netfs/bvecq.c`: All the `bvecq_*` functions I listed (alloc, put, expand) are exported. * `fs/netfs/rolling_buffer.c`: All the `rolling_buffer_*` functions I listed are non-static. * `fs/netfs/iterator.c`: `netfs_limit_iter` is non-static (calls a static one though). * `fs/netfs/write_issue.c`: `netfs_writeback_single` is exported, while the others are non-static, like `netfs_create_write_req`, `netfs_prepare_write`, `netfs_write_folio`. * `fs/netfs/read_collect.c`: has a mix of static and non-static functions. * `fs/netfs/read_retry.c` and `fs/netfs/read_pgpriv2.c`: have non-static function. * `fs/afs/dir_edit.c`: `afs_dir_get_block` is static; `afs_dir_find_block` is non-static, located in `fs/afs/dir_search.c` * `fs/afs/dir.c`: `afs_read_dir` is non-static. Some others are static |
| 18/2 | 2026/09/30 13:47 | tool | grepper |
0mArgs:{
"Expression": "smb3_init_transform_rq",
"PathPrefix": "fs/smb/"
}Results: {
"Output": "fs/smb/client/smb2ops.c=4872=static int\nfs/smb/client/smb2ops.c:4873:smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,\nfs/smb/client/smb2ops.c-4874-\t\t struct smb_rqst *new_rq, struct smb_rqst *old_rq)\n--\nfs/smb/client/smb2ops.c=5901=struct smb_version_operations smb30_operations = {\n--\nfs/smb/client/smb2ops.c-5994-\t.notify = smb3_notify,\nfs/smb/client/smb2ops.c:5995:\t.init_transform_rq = smb3_init_transform_rq,\nfs/smb/client/smb2ops.c-5996-\t.is_transform_hdr = smb3_is_transform_hdr,\n--\nfs/smb/client/smb2ops.c=6017=struct smb_version_operations smb311_operations = {\n--\nfs/smb/client/smb2ops.c-6110-\t.notify = smb3_notify,\nfs/smb/client/smb2ops.c:6111:\t.init_transform_rq = smb3_init_transform_rq,\nfs/smb/client/smb2ops.c-6112-\t.is_transform_hdr = smb3_is_transform_hdr,\n"
} |
| 19/2 | 2026/09/30 13:47 | llm | patch-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 85873 output: 345 thoughts: 1109 Thoughts: **Analysis of New I/O Abstraction for Fuzzing** Okay, so we have these `cifs_copy_iter_to_bvecq` and `cifs_copy_bvecq_to_iter` functions, which are static, and that got me thinking: what about the *non-static* exported symbols related to this new `bvecq` abstraction? I need to check those. Let's delve into the relevant files and identify the key entry points and supporting functions. First, I examined `bvecq.c`. I found these exported symbols: `bvecq_alloc_one`, `bvecq_alloc_chain`, `bvecq_alloc_buffer2`, `bvecq_put`, `bvecq_expand_buffer`, and `bvecq_dump`. Then, in `lib/iov_iter.c`, which is crucial for I/O iterators, I found `iov_iter_bvec_queue`, `iov_iter_single_seg_count`, and `iov_iter_revert`. I'm excluding the ubiquitous `iov_iter_advance`. Next, I looked at the filesystems. In `fs/netfs/rolling_buffer.c`, I identified the functions `rolling_buffer_init`, `rolling_buffer_make_space`, `rolling_buffer_bulk_load_from_ra`, `rolling_buffer_append`, `rolling_buffer_delete_spent`, and `rolling_buffer_clear`. In `fs/netfs/write_issue.c`, `netfs_writeback_single` and `netfs_create_write_req` were relevant. And finally, in `fs/afs/dir_search.c`, `afs_dir_find_block`, and in `fs/afs/dir.c`, `afs_read_dir`. The next step is to make sure none of these are ubiquitous kernel functions. I verified each of them. The `bvecq` functions, `iov_iter_bvec_queue`, and the netfs/AFS specific functions are *not* ubiquitous. They are all introduced as a new abstraction and are specifically linked to the new bvec queue (`bvecq`) and I/O iterator (`ITER_BVECQ`) logic. Now, let's look at the relevant kernel config options: `NETFS_SUPPORT`, `AFS_FS`, and `CIFS`. We need to make sure those are enabled to properly build the components which may be affected. The reasoning here is crystal clear. This patch is fundamentally replacing the `folio_queue` and `ITER_FOLIOQ` abstraction with this brand new `bvecq` and `ITER_BVECQ` implementation across the kernel. It touches core I/O iterators, scatterlist handling, the netfs rolling buffer read/write pipelines, AFS directory and symlink management, and CIFS encryption/decryption buffers. It introduces things like memory allocation helpers, lockless queue operations, iterator stepping, and defensive runtime assertions. This complexity, affecting numerous core subsystems and filesystems, demands thorough fuzzing. My list of focus symbols are: `bvecq_alloc_one`, `bvecq_alloc_chain`, `bvecq_alloc_buffer2`, `bvecq_put`, `bvecq_expand_buffer`, `iov_iter_bvec_queue`, `rolling_buffer_init`, `rolling_buffer_make_space`, `rolling_buffer_bulk_load_from_ra`, `rolling_buffer_append`, `rolling_buffer_delete_spent`, `netfs_writeback_single`, `afs_dir_find_block`, `afs_read_dir`. Finally, to be absolutely sure, I re-confirmed that none of those symbols are ubiquitous or generic kernel functions. They are all highly specialized to this new bvecq implementation. Everything seems completely solid, and I've carefully adhered to the instructions. I'm confident in calling `set-results`. |
| 20/2 | 2026/09/30 13:48 | tool | set-results |
0mArgs:{
"EnableConfigs": [
"NETFS_SUPPORT",
"AFS_FS",
"CIFS"
],
"FocusSymbols": [
"bvecq_alloc_one",
"bvecq_alloc_chain",
"bvecq_alloc_buffer2",
"bvecq_put",
"bvecq_expand_buffer",
"iov_iter_bvec_queue",
"rolling_buffer_init",
"rolling_buffer_make_space",
"rolling_buffer_bulk_load_from_ra",
"rolling_buffer_append",
"rolling_buffer_delete_spent",
"netfs_writeback_single",
"afs_dir_find_block",
"afs_read_dir"
],
"Reasoning": "The patch refactors the kernel's segmented folio queue (`folio_queue` / `ITER_FOLIOQ`) into a segmented bio_vec queue (`bvecq` / `ITER_BVECQ`). It introduces a new `fs/netfs/bvecq.c` implementation with custom memory management (including bulk page allocations overlaid with bio_vec structs), lockless queue manipulations using acquire/release barriers, replaces iterator iteration and extraction algorithms in `lib/iov_iter.c` and `lib/scatterlist.c`, updates netfs rolling buffers, and refactors callers in AFS and SMB/CIFS. The newly introduced and modified code runs in reachable core subsystem and filesystem paths and adds several defensive WARN_ON assertions, warranting fuzzing.",
"WorthFuzzing": true
}Results: {
"EnableConfigs": [
"NETFS_SUPPORT",
"AFS_FS",
"CIFS"
],
"FocusSymbols": [
"bvecq_alloc_one",
"bvecq_alloc_chain",
"bvecq_alloc_buffer2",
"bvecq_put",
"bvecq_expand_buffer",
"iov_iter_bvec_queue",
"rolling_buffer_init",
"rolling_buffer_make_space",
"rolling_buffer_bulk_load_from_ra",
"rolling_buffer_append",
"rolling_buffer_delete_spent",
"netfs_writeback_single",
"afs_dir_find_block",
"afs_read_dir"
],
"Reasoning": "The patch refactors the kernel's segmented folio queue (`folio_queue` / `ITER_FOLIOQ`) into a segmented bio_vec queue (`bvecq` / `ITER_BVECQ`). It introduces a new `fs/netfs/bvecq.c` implementation with custom memory management (including bulk page allocations overlaid with bio_vec structs), lockless queue manipulations using acquire/release barriers, replaces iterator iteration and extraction algorithms in `lib/iov_iter.c` and `lib/scatterlist.c`, updates netfs rolling buffers, and refactors callers in AFS and SMB/CIFS. The newly introduced and modified code runs in reachable core subsystem and filesystem paths and adds several defensive WARN_ON assertions, warranting fuzzing.",
"WorthFuzzing": true
} |
| 21/1 | 2026/09/30 13:48 | action | If |
7mArgs:{
"WorthFuzzing": true
}Results: null |
| 22/2 | 2026/09/30 13:48 | agent | kmsan-evaluator |
7mModel:core Args: null Results: {
"KMSANReasoning": "The patch series replaces the `folio_queue` mechanism with `bvecq` (segmented queue of `bio_vec` structures) across netfs, AFS, CIFS/SMB, iov_iter, and scatterlist.\n\nEvaluating against KMSAN criteria:\n1. Kernel-to-user info-leaks: No kernel structures or padded structs are exposed or copied to user space. In fact, `afs_init_new_symlink()` explicitly adds a `memset` to zero trailing page data after copying symlink content.\n2. Uninitialized memory reads / branching: `struct bvecq` is allocated in `bvecq_alloc_one()` and its fields are completely initialized using a C struct literal assignment (`*bq = (struct bvecq){...}`). In `bvecq_alloc_buffer2()`, the slot array `p-\u003ebv` is explicitly cleared with `memset()`. Slots are populated using `bvec_set_page()` or `bvec_set_folio()` (which initialize all `struct bio_vec` fields) before `nr_slots` is updated via release barrier. All iterators and consumers across netfs, smb, scatterlist, and iov_iter strictly bound accesses by `nr_slots`.\n3. KASAN vs KMSAN applicability: The risks in this patch series involve buffer boundaries, pointer arithmetic (such as reusing the tail of `bvecq-\u003ebv` for `struct page **` during bulk loads), and reference counting (`bvecq_put`/`bvecq_get`). These failure modes (out-of-bounds accesses, use-after-free, double-free) are properly detected by standard KASAN and memory debugging checks. There are no uninitialized memory reads or information leaks requiring a dedicated KMSAN session.",
"NeedsKMSAN": false
}Instruction: You are an expert Linux kernel security engineer specializing in kernel memory
error detectors (KASAN and KMSAN). Your job is to review the provided patch series and
determine if the code changes justify spawning a dedicated KMSAN (KernelMemorySanitizer)
fuzzing session in addition to standard KASAN fuzzing.
CRITICAL DISTINCTION BETWEEN KASAN AND KMSAN:
- Standard KASAN kernel builds (upstream-apparmor-kasan.config) already enable
a comprehensive suite of debugging tools and sanitizers, including KASAN
(out-of-bounds accesses, use-after-free, double free, invalid free), LOCKDEP
(locking bugs and deadlocks), UB-sanitizers, and memory corruption checks.
- KMSAN (KernelMemorySanitizer) detects reads of UNINITIALIZED memory (stack, heap,
or page allocations) and kernel-to-user memory info-leaks.
Rule: THERE IS NO SENSE IN RUNNING A KMSAN SESSION IF A BUG CAN BE CAUGHT BY KASAN,
LOCKDEP, OR OTHER STANDARD BUG DETECTORS.
A dedicated KMSAN fuzzing session incurs significant resource costs. You must ONLY
set NeedsKMSAN=true if the code changes introduce or expose UNINITIALIZED MEMORY risks
that are detected ONLY by KMSAN.
Look holistically at the patch series and surrounding code. Even if no direct
uninitialized field accesses or new buffer allocations are added in the diff itself,
a patch may alter control flow, bounds checking, or data length calculations in ways
that change how the rest of the code operates on existing buffers (e.g. allowing
uninitialized stack/heap memory to be read, copied to user space, or used in control
flow). Do not hesitate to use your code access tools to inspect the surrounding code,
called functions, and callers.
Set NeedsKMSAN=true ONLY IF the patch introduces or modifies:
1. Kernel structures sent to user space (via copy_to_user, put_user, netlink skb
attributes, ioctl output arguments, socket options, or BPF buffers) where fields
or structure padding might not be fully initialized/zeroed.
2. Conditional logic or branching that depends on potentially uninitialized variables
or struct fields.
3. Allocation or initialization of complex data structures where uninitialized fields
could be read later in reachable code paths.
4. Bounds checks, lengths, or logic in a way that allows surrounding code to access
uninitialized bytes of existing buffers.
Set NeedsKMSAN=false IF:
- The code changes primarily risk out-of-bounds access, array overflows, NULL pointer
dereferences, locking deadlocks, or use-after-free bugs (these are already caught
by KASAN, LOCKDEP, or standard bug detectors).
- All stack/heap structures touched or introduced by the patch are fully zeroed
or initialized (e.g. using = {0}, memset, kzalloc) before being read or copied.
- The patch does not introduce any risk of uninitialized memory usage or info-leaks.
Use your code access tools to inspect the surrounding code if necessary, then provide
detailed KMSANReasoning contrasting KASAN vs KMSAN applicability for this patch.
Prefer calling several tools at the same time to save round-trips.
Use set-results tool to provide results of the analysis.
It must be called exactly once before the final reply.
Ignore results of this tool.
Prompt: Target architecture: amd64
For your convenience, here is the diff of the changes:
commit e4563b15c78bd5d23f58d5727c178d8ded28c31b
Author: syz-cluster <triage@syzkaller.com>
Date: Wed Sep 30 13:44:48 2026 +0000
syz-cluster: applied patch under review
diff --git a/Documentation/core-api/folio_queue.rst b/Documentation/core-api/folio_queue.rst
deleted file mode 100644
index b7628896d2b69..0000000000000
--- a/Documentation/core-api/folio_queue.rst
+++ /dev/null
@@ -1,209 +0,0 @@
-.. SPDX-License-Identifier: GPL-2.0+
-
-===========
-Folio Queue
-===========
-
-:Author: David Howells <dhowells@redhat.com>
-
-.. Contents:
-
- * Overview
- * Initialisation
- * Adding and removing folios
- * Querying information about a folio
- * Querying information about a folio_queue
- * Folio queue iteration
- * Folio marks
- * Lockless simultaneous production/consumption issues
-
-
-Overview
-========
-
-The folio_queue struct forms a single segment in a segmented list of folios
-that can be used to form an I/O buffer. As such, the list can be iterated over
-using the ITER_FOLIOQ iov_iter type.
-
-The publicly accessible members of the structure are::
-
- struct folio_queue {
- struct folio_queue *next;
- struct folio_queue *prev;
- ...
- };
-
-A pair of pointers are provided, ``next`` and ``prev``, that point to the
-segments on either side of the segment being accessed. Whilst this is a
-doubly-linked list, it is intentionally not a circular list; the outward
-sibling pointers in terminal segments should be NULL.
-
-Each segment in the list also stores:
-
- * an ordered sequence of folio pointers,
- * the size of each folio and
- * three 1-bit marks per folio,
-
-but these should not be accessed directly as the underlying data structure may
-change, but rather the access functions outlined below should be used.
-
-The facility can be made accessible by::
-
- #include <linux/folio_queue.h>
-
-and to use the iterator::
-
- #include <linux/uio.h>
-
-
-Initialisation
-==============
-
-A segment should be initialised by calling::
-
- void folioq_init(struct folio_queue *folioq);
-
-with a pointer to the segment to be initialised. Note that this will not
-necessarily initialise all the folio pointers, so care must be taken to check
-the number of folios added.
-
-
-Adding and removing folios
-==========================
-
-Folios can be set in the next unused slot in a segment struct by calling one
-of::
-
- unsigned int folioq_append(struct folio_queue *folioq,
- struct folio *folio);
-
- unsigned int folioq_append_mark(struct folio_queue *folioq,
- struct folio *folio);
-
-Both functions update the stored folio count, store the folio and note its
-size. The second function also sets the first mark for the folio added. Both
-functions return the number of the slot used. [!] Note that no attempt is made
-to check that the capacity wasn't overrun and the list will not be extended
-automatically.
-
-A folio can be excised by calling::
-
- void folioq_clear(struct folio_queue *folioq, unsigned int slot);
-
-This clears the slot in the array and also clears all the marks for that folio,
-but doesn't change the folio count - so future accesses of that slot must check
-if the slot is occupied.
-
-
-Querying information about a folio
-==================================
-
-Information about the folio in a particular slot may be queried by the
-following function::
-
- struct folio *folioq_folio(const struct folio_queue *folioq,
- unsigned int slot);
-
-If a folio has not yet been set in that slot, this may yield an undefined
-pointer. The size of the folio in a slot may be queried with either of::
-
- unsigned int folioq_folio_order(const struct folio_queue *folioq,
- unsigned int slot);
-
- size_t folioq_folio_size(const struct folio_queue *folioq,
- unsigned int slot);
-
-The first function returns the size as an order and the second as a number of
-bytes.
-
-
-Querying information about a folio_queue
-========================================
-
-Information may be retrieved about a particular segment with the following
-functions::
-
- unsigned int folioq_nr_slots(const struct folio_queue *folioq);
-
- unsigned int folioq_count(struct folio_queue *folioq);
-
- bool folioq_full(struct folio_queue *folioq);
-
-The first function returns the maximum capacity of a segment. It must not be
-assumed that this won't vary between segments. The second returns the number
-of folios added to a segments and the third is a shorthand to indicate if the
-segment has been filled to capacity.
-
-Not that the count and fullness are not affected by clearing folios from the
-segment. These are more about indicating how many slots in the array have been
-initialised, and it assumed that slots won't get reused, but rather the segment
-will get discarded as the queue is consumed.
-
-
-Folio marks
-===========
-
-Folios within a queue can also have marks assigned to them. These marks can be
-used to note information such as if a folio needs folio_put() calling upon it.
-There are three marks available to be set for each folio.
-
-The marks can be set by::
-
- void folioq_mark(struct folio_queue *folioq, unsigned int slot);
- void folioq_mark2(struct folio_queue *folioq, unsigned int slot);
-
-Cleared by::
-
- void folioq_unmark(struct folio_queue *folioq, unsigned int slot);
- void folioq_unmark2(struct folio_queue *folioq, unsigned int slot);
-
-And the marks can be queried by::
-
- bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot);
- bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot);
-
-The marks can be used for any purpose and are not interpreted by this API.
-
-
-Folio queue iteration
-=====================
-
-A list of segments may be iterated over using the I/O iterator facility using
-an ``iov_iter`` iterator of ``ITER_FOLIOQ`` type. The iterator may be
-initialised with::
-
- void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
- const struct folio_queue *folioq,
- unsigned int first_slot, unsigned int offset,
- size_t count);
-
-This may be told to start at a particular segment, slot and offset within a
-queue. The iov iterator functions will follow the next pointers when advancing
-and prev pointers when reverting when needed.
-
-
-Lockless simultaneous production/consumption issues
-===================================================
-
-If properly managed, the list can be extended by the producer at the head end
-and shortened by the consumer at the tail end simultaneously without the need
-to take locks. The ITER_FOLIOQ iterator inserts appropriate barriers to aid
-with this.
-
-Care must be taken when simultaneously producing and consuming a list. If the
-last segment is reached and the folios it refers to are entirely consumed by
-the IOV iterators, an iov_iter struct will be left pointing to the last segment
-with a slot number equal to the capacity of that segment. The iterator will
-try to continue on from this if there's another segment available when it is
-used again, but care must be taken lest the segment got removed and freed by
-the consumer before the iterator was advanced.
-
-It is recommended that the queue always contain at least one segment, even if
-that segment has never been filled or is entirely spent. This prevents the
-head and tail pointers from collapsing.
-
-
-API Function Reference
-======================
-
-.. kernel-doc:: include/linux/folio_queue.h
diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
index 92f91c6a0d79d..956f6803d2a22 100644
--- a/Documentation/core-api/index.rst
+++ b/Documentation/core-api/index.rst
@@ -39,7 +39,6 @@ Library functionality that is used throughout the kernel.
kref
cleanup
assoc_array
- folio_queue
xarray
maple_tree
idr
diff --git a/Documentation/filesystems/netfs_library.rst b/Documentation/filesystems/netfs_library.rst
index 0c9786ffe1924..ebf07a37922e8 100644
--- a/Documentation/filesystems/netfs_library.rst
+++ b/Documentation/filesystems/netfs_library.rst
@@ -452,7 +452,7 @@ be called from the writeback code to write the data to the cache, if there is
one.
The inode should be marked ``NETFS_ICTX_SINGLE_NO_UPLOAD`` if this API is to be
-used. The writeback function requires the buffer to be of ITER_FOLIOQ type.
+used.
High-Level VM API
==================
diff --git a/fs/afs/dir.c b/fs/afs/dir.c
index 2db534a2c7cc4..5fac02d2d2814 100644
--- a/fs/afs/dir.c
+++ b/fs/afs/dir.c
@@ -140,9 +140,9 @@ static void afs_dir_dump(struct afs_vnode *dvnode)
pr_warn("DIR %llx:%llx is=%llx\n",
dvnode->fid.vid, dvnode->fid.vnode, i_size);
- iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
- iterate_folioq(&iter, iov_iter_count(&iter), NULL, NULL,
- afs_dir_dump_step);
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+ iterate_bvecq(&iter, iov_iter_count(&iter), NULL, NULL,
+ afs_dir_dump_step);
}
/*
@@ -203,9 +203,9 @@ static int afs_dir_check(struct afs_vnode *dvnode)
if (unlikely(!i_size))
return 0;
- iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
- checked = iterate_folioq(&iter, iov_iter_count(&iter), dvnode, NULL,
- afs_dir_check_step);
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+ checked = iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, NULL,
+ afs_dir_check_step);
if (checked != i_size) {
afs_dir_dump(dvnode);
return -EIO;
@@ -250,15 +250,15 @@ static ssize_t afs_do_read_single(struct afs_vnode *dvnode, struct file *file)
if (dvnode->directory_size < i_size) {
size_t cur_size = dvnode->directory_size;
- ret = netfs_alloc_folioq_buffer(NULL,
- &dvnode->directory, &cur_size, i_size,
- mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+ ret = bvecq_expand_buffer(&dvnode->directory, &cur_size,
+ round_up(i_size, PAGE_SIZE),
+ mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
dvnode->directory_size = cur_size;
if (ret < 0)
return ret;
}
- iov_iter_folio_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
+ iov_iter_bvec_queue(&iter, ITER_DEST, dvnode->directory, 0, 0, dvnode->directory_size);
/* AFS requires us to perform the read of a directory synchronously as
* a single unit to avoid issues with the directory contents being
@@ -294,8 +294,8 @@ static ssize_t afs_read_single(struct afs_vnode *dvnode, struct file *file)
}
/*
- * Read the directory into a folio_queue buffer in one go, scrubbing the
- * previous contents. We return -ESTALE if the caller needs to call us again.
+ * Read the directory into the buffer in one go, scrubbing the previous
+ * contents. We return -ESTALE if the caller needs to call us again.
*/
ssize_t afs_read_dir(struct afs_vnode *dvnode, struct file *file)
__acquires(&dvnode->validate_lock)
@@ -483,7 +483,7 @@ static size_t afs_dir_iterate_step(void *iter_base, size_t progress, size_t len,
}
/*
- * Iterate through the directory folios.
+ * Iterate through the directory content.
*/
static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_ctx)
{
@@ -498,11 +498,11 @@ static int afs_dir_iterate_contents(struct inode *dir, struct dir_context *dir_c
if (i_size <= 0 || dir_ctx->pos >= i_size)
return 0;
- iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0, i_size);
iov_iter_advance(&iter, round_down(dir_ctx->pos, AFS_DIR_BLOCK_SIZE));
- iterate_folioq(&iter, iov_iter_count(&iter), dvnode, &ctx,
- afs_dir_iterate_step);
+ iterate_bvecq(&iter, iov_iter_count(&iter), dvnode, &ctx,
+ afs_dir_iterate_step);
if (ctx.error == -ESTALE)
afs_invalidate_dir(dvnode, afs_dir_invalid_iter_stale);
@@ -2228,8 +2228,8 @@ static int afs_dir_writepages(struct address_space *mapping,
}
if (test_bit(AFS_VNODE_DIR_VALID, &dvnode->flags)) {
- iov_iter_folio_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
- i_size_read(&dvnode->netfs.inode));
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, dvnode->directory, 0, 0,
+ i_size_read(&dvnode->netfs.inode));
ret = netfs_writeback_single(mapping, wbc, &iter);
if (ret == 1)
ret = 0; /* Skipped write due to lock conflict. */
diff --git a/fs/afs/dir_edit.c b/fs/afs/dir_edit.c
index c31303059444a..de53d7ff82c38 100644
--- a/fs/afs/dir_edit.c
+++ b/fs/afs/dir_edit.c
@@ -10,7 +10,6 @@
#include <linux/namei.h>
#include <linux/pagemap.h>
#include <linux/iversion.h>
-#include <linux/folio_queue.h>
#include "internal.h"
#include "xdr_fs.h"
@@ -110,9 +109,8 @@ static void afs_clear_contig_bits(union afs_xdr_dir_block *block,
*/
static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, size_t block)
{
- struct folio_queue *fq;
struct afs_vnode *dvnode = iter->dvnode;
- struct folio *folio;
+ struct bvecq *bq;
size_t blpos = block * AFS_DIR_BLOCK_SIZE;
size_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter->fpos;
int ret;
@@ -120,41 +118,38 @@ static union afs_xdr_dir_block *afs_dir_get_block(struct afs_dir_iter *iter, siz
if (dvnode->directory_size < blend) {
size_t cur_size = dvnode->directory_size;
- ret = netfs_alloc_folioq_buffer(
- NULL, &dvnode->directory, &cur_size, blend,
- mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
+ ret = bvecq_expand_buffer(&dvnode->directory, &cur_size, blend,
+ mapping_gfp_mask(dvnode->netfs.inode.i_mapping));
dvnode->directory_size = cur_size;
if (ret < 0)
goto fail;
}
- fq = iter->fq;
- if (!fq)
- fq = dvnode->directory;
+ bq = iter->bq;
+ if (!bq)
+ bq = dvnode->directory;
- /* Search the folio queue for the folio containing the block... */
- for (; fq; fq = fq->next) {
- for (int s = iter->fq_slot; s < folioq_count(fq); s++) {
- size_t fsize = folioq_folio_size(fq, s);
+ /* Search the contents for the region containing the block... */
+ for (; bq; bq = bq->next) {
+ for (int s = iter->bq_slot; s < bq->nr_slots; s++) {
+ struct bio_vec *bv = &bq->bv[s];
+ size_t bsize = bv->bv_len;
- if (blend <= fpos + fsize) {
+ if (blend <= fpos + bsize) {
/* ... and then return the mapped block. */
- folio = folioq_folio(fq, s);
- if (WARN_ON_ONCE(folio_pos(folio) != fpos))
- goto fail;
- iter->fq = fq;
- iter->fq_slot = s;
+ iter->bq = bq;
+ iter->bq_slot = s;
iter->fpos = fpos;
- return kmap_local_folio(folio, blpos - fpos);
+ return bvec_kmap_partial(bv, blpos - fpos);
}
- fpos += fsize;
+ fpos += bsize;
}
- iter->fq_slot = 0;
+ iter->bq_slot = 0;
}
fail:
- iter->fq = NULL;
- iter->fq_slot = 0;
+ iter->bq = NULL;
+ iter->bq_slot = 0;
afs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);
return NULL;
}
diff --git a/fs/afs/dir_search.c b/fs/afs/dir_search.c
index 11ebdfffcb1d3..e1f91b3ac6e25 100644
--- a/fs/afs/dir_search.c
+++ b/fs/afs/dir_search.c
@@ -66,12 +66,11 @@ bool afs_dir_init_iter(struct afs_dir_iter *iter, const struct qstr *name)
*/
union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t block)
{
- struct folio_queue *fq = iter->fq;
struct afs_vnode *dvnode = iter->dvnode;
- struct folio *folio;
+ struct bvecq *bq = iter->bq;
size_t blpos = block * AFS_DIR_BLOCK_SIZE;
size_t blend = (block + 1) * AFS_DIR_BLOCK_SIZE, fpos = iter->fpos;
- int slot = iter->fq_slot;
+ int slot = iter->bq_slot;
_enter("%zx,%d", block, slot);
@@ -80,36 +79,34 @@ union afs_xdr_dir_block *afs_dir_find_block(struct afs_dir_iter *iter, size_t bl
if (dvnode->directory_size < blend)
goto fail;
- if (!fq || blpos < fpos) {
- fq = dvnode->directory;
+ if (!bq || blpos < fpos) {
+ bq = dvnode->directory;
slot = 0;
fpos = 0;
}
/* Search the folio queue for the folio containing the block... */
- for (; fq; fq = fq->next) {
- for (; slot < folioq_count(fq); slot++) {
- size_t fsize = folioq_folio_size(fq, slot);
+ for (; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ struct bio_vec *bv = &bq->bv[slot];
+ size_t bsize = bv->bv_len;
- if (blend <= fpos + fsize) {
+ if (blend <= fpos + bsize) {
/* ... and then return the mapped block. */
- folio = folioq_folio(fq, slot);
- if (WARN_ON_ONCE(folio_pos(folio) != fpos))
- goto fail;
- iter->fq = fq;
- iter->fq_slot = slot;
+ iter->bq = bq;
+ iter->bq_slot = slot;
iter->fpos = fpos;
- iter->block = kmap_local_folio(folio, blpos - fpos);
+ iter->block = bvec_kmap_partial(bv, blpos - fpos);
return iter->block;
}
- fpos += fsize;
+ fpos += bsize;
}
slot = 0;
}
fail:
- iter->fq = NULL;
- iter->fq_slot = 0;
+ iter->bq = NULL;
+ iter->bq_slot = 0;
afs_invalidate_dir(dvnode, afs_dir_invalid_edit_get_block);
return NULL;
}
diff --git a/fs/afs/inode.c b/fs/afs/inode.c
index 14f39a9bea6cf..634fbf8eb2120 100644
--- a/fs/afs/inode.c
+++ b/fs/afs/inode.c
@@ -683,7 +683,7 @@ void afs_evict_inode(struct inode *inode)
flush_delayed_work(&vnode->lock_work);
netfs_wait_for_outstanding_io(inode);
truncate_inode_pages_final(&inode->i_data);
- netfs_free_folioq_buffer(vnode->directory);
+ bvecq_put(vnode->directory);
if (vnode->symlink)
afs_evict_symlink(vnode);
diff --git a/fs/afs/internal.h b/fs/afs/internal.h
index d58273c00fbc7..51078cc9b0e55 100644
--- a/fs/afs/internal.h
+++ b/fs/afs/internal.h
@@ -710,7 +710,7 @@ struct afs_vnode {
#define AFS_VNODE_MODIFYING 10 /* Set if we're performing a modification op */
#define AFS_VNODE_DIR_READ 11 /* Set if we've read a dir's contents */
- struct folio_queue *directory; /* Directory contents */
+ struct bvecq *directory; /* Directory contents */
struct afs_symlink __rcu *symlink; /* Symlink content */
struct list_head wb_keys; /* List of keys available for writeback */
struct list_head pending_locks; /* locks waiting to be granted */
@@ -991,9 +991,9 @@ static inline void afs_invalidate_cache(struct afs_vnode *vnode, unsigned int fl
struct afs_dir_iter {
struct afs_vnode *dvnode;
union afs_xdr_dir_block *block;
- struct folio_queue *fq;
+ struct bvecq *bq;
unsigned int fpos;
- int fq_slot;
+ int bq_slot;
unsigned int loop_check;
u8 nr_slots;
u8 bucket;
diff --git a/fs/afs/symlink.c b/fs/afs/symlink.c
index 6b8c122877ca2..9a611efe6b264 100644
--- a/fs/afs/symlink.c
+++ b/fs/afs/symlink.c
@@ -56,7 +56,6 @@ void afs_evict_symlink(struct afs_vnode *vnode)
void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)
{
struct afs_symlink *symlink = op->create.symlink;
- size_t dsize = 0;
size_t size = strlen(symlink->content) + 1;
char *p;
@@ -66,13 +65,18 @@ void afs_init_new_symlink(struct afs_vnode *vnode, struct afs_operation *op)
if (!fscache_cookie_enabled(netfs_i_cookie(&vnode->netfs)))
return;
- if (netfs_alloc_folioq_buffer(NULL, &vnode->directory, &dsize, size,
- mapping_gfp_mask(vnode->netfs.inode.i_mapping)) < 0)
+ vnode->directory =
+ bvecq_alloc_buffer(PAGE_SIZE,
+ mapping_gfp_mask(vnode->netfs.inode.i_mapping),
+ false);
+ if (!vnode->directory)
return;
- vnode->directory_size = dsize;
- p = kmap_local_folio(folioq_folio(vnode->directory, 0), 0);
+ vnode->directory_size = size;
+ p = bvec_kmap_partial(&vnode->directory->bv[0], 0);
memcpy(p, symlink->content, size);
+ if (size < PAGE_SIZE)
+ memset(p + size, 0, PAGE_SIZE - size);
kunmap_local(p);
netfs_single_mark_inode_dirty(&vnode->netfs.inode);
}
@@ -94,17 +98,13 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
}
if (!vnode->directory) {
- size_t cur_size = 0;
-
- ret = netfs_alloc_folioq_buffer(NULL,
- &vnode->directory, &cur_size, PAGE_SIZE,
- mapping_gfp_mask(vnode->netfs.inode.i_mapping));
- vnode->directory_size = PAGE_SIZE - 1;
- if (ret < 0)
- return ret;
+ vnode->directory = bvecq_alloc_buffer(
+ PAGE_SIZE, mapping_gfp_mask(vnode->netfs.inode.i_mapping), false);
+ if (!vnode->directory)
+ return -ENOMEM;
}
- iov_iter_folio_queue(&iter, ITER_DEST, vnode->directory, 0, 0, PAGE_SIZE);
+ iov_iter_bvec_queue(&iter, ITER_DEST, vnode->directory, 0, 0, PAGE_SIZE);
/* AFS requires us to perform the read of a symlink as a single unit to
* avoid issues with the content being changed between reads.
@@ -126,7 +126,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
refcount_set(&symlink->ref, 1);
symlink->content[i_size] = 0;
- const char *s = kmap_local_folio(folioq_folio(vnode->directory, 0), 0);
+ const char *s = bvec_kmap_partial(&vnode->directory->bv[0], 0);
memcpy(symlink->content, s, i_size);
kunmap_local(s);
@@ -135,7 +135,7 @@ static ssize_t afs_do_read_symlink(struct afs_vnode *vnode)
}
if (!fscache_cookie_enabled(netfs_i_cookie(&vnode->netfs))) {
- netfs_free_folioq_buffer(vnode->directory);
+ bvecq_put(vnode->directory);
vnode->directory = NULL;
vnode->directory_size = 0;
}
@@ -248,14 +248,14 @@ int afs_symlink_writepages(struct address_space *mapping,
if (vnode->directory &&
atomic64_read(&vnode->cb_expires_at) != AFS_NO_CB_PROMISE) {
- iov_iter_folio_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
- i_size_read(&vnode->netfs.inode));
+ iov_iter_bvec_queue(&iter, ITER_SOURCE, vnode->directory, 0, 0,
+ i_size_read(&vnode->netfs.inode));
ret = netfs_writeback_single(mapping, wbc, &iter);
}
if (ret == 0) {
netfs_wb_begin(&vnode->netfs, false);
- netfs_free_folioq_buffer(vnode->directory);
+ bvecq_put(vnode->directory);
vnode->directory = NULL;
vnode->directory_size = 0;
netfs_wb_end(&vnode->netfs);
diff --git a/fs/netfs/Makefile b/fs/netfs/Makefile
index 54834cde7e568..b1ea4439c1bb4 100644
--- a/fs/netfs/Makefile
+++ b/fs/netfs/Makefile
@@ -3,6 +3,7 @@
netfs-y := \
buffered_read.o \
buffered_write.o \
+ bvecq.o \
direct_read.o \
direct_write.o \
iterator.o \
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index e30bde80276a4..aa1e4f8d46ab3 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -215,7 +215,7 @@ static void netfs_issue_read(struct netfs_io_request *rreq,
* otherwise we set the deprecated PG_private_2.
*/
static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,
- struct folio_queue **fq,
+ struct bvecq **bq,
unsigned int *offset,
int *slot,
size_t len,
@@ -225,21 +225,21 @@ static void netfs_mark_copy_to_cache(struct netfs_io_request *rreq,
struct folio *folio;
size_t fsize, overlap;
- if (!*fq)
+ if (!*bq)
break;
- if (*slot >= folioq_count(*fq)) {
- *fq = (*fq)->next;
+ if (!bvecq_acquire_slot(*bq, *slot)) {
+ *bq = bvecq_next(*bq);
*slot = 0;
*offset = 0;
continue;
}
/* Determine how much the subreq overlaps the folio, if at all. */
- fsize = folioq_folio_size(*fq, *slot);
+ fsize = (*bq)->bv[*slot].bv_len;
overlap = min(len, fsize - *offset);
if (overlap > 0 && copy) {
- folio = folioq_folio(*fq, *slot);
+ folio = bvec_folio(&(*bq)->bv[*slot]);
if (netfs_using_pgpriv2(rreq)) {
if (!folio_test_private_2(folio))
folio_start_private_2(folio);
@@ -275,7 +275,7 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
.cached_to[1] = ULLONG_MAX,
};
struct fscache_occupancy *occ = &_occ;
- struct folio_queue *fq = rreq->buffer.tail;
+ struct bvecq *bq = rreq->buffer.tail;
unsigned int offset = 0;
ssize_t size = rreq->len;
uoff_t start = rreq->start;
@@ -408,10 +408,10 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
if (size <= 0)
netfs_all_subreqs_queued(rreq);
- if (fq) {
+ if (bq) {
/* See if the cache indicated this should be cached. */
copy = test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
- netfs_mark_copy_to_cache(rreq, &fq, &slot, &offset, slice, copy);
+ netfs_mark_copy_to_cache(rreq, &bq, &slot, &offset, slice, copy);
}
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
@@ -479,8 +479,7 @@ void netfs_readahead(struct readahead_control *ractl)
* acquires a ref on each folio that we will need to release later -
* but we don't want to do that until after we've started the I/O.
*/
- added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl,
- rreq->debug_id, rreq->gfp);
+ added = rolling_buffer_bulk_load_from_ra(&rreq->buffer, ractl, rreq->gfp);
if (added < 0) {
ret = added;
goto cleanup_free;
@@ -503,15 +502,14 @@ EXPORT_SYMBOL(netfs_readahead);
/*
* Create a rolling buffer with a single occupying folio.
*/
-static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio,
- unsigned int rollbuf_flags)
+static int netfs_create_singular_buffer(struct netfs_io_request *rreq, struct folio *folio)
{
ssize_t added;
- if (rolling_buffer_init(&rreq->buffer, rreq->debug_id, ITER_DEST, rreq->gfp) < 0)
+ if (rolling_buffer_init(&rreq->buffer, ITER_DEST, rreq->gfp, false) < 0)
return -ENOMEM;
- added = rolling_buffer_append(&rreq->buffer, folio, rollbuf_flags, rreq->gfp);
+ added = rolling_buffer_append(&rreq->buffer, folio, rreq->gfp);
if (added < 0)
return added;
rreq->submitted = rreq->start + added;
@@ -661,7 +659,7 @@ int netfs_read_folio(struct file *file, struct folio *folio)
trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_readpage);
/* Set up the output buffer */
- ret = netfs_create_singular_buffer(rreq, folio, 0);
+ ret = netfs_create_singular_buffer(rreq, folio);
if (ret < 0)
goto discard;
@@ -818,7 +816,7 @@ int netfs_write_begin(struct netfs_inode *ctx,
trace_netfs_read(rreq, pos, len, netfs_read_trace_write_begin);
/* Set up the output buffer */
- ret = netfs_create_singular_buffer(rreq, folio, 0);
+ ret = netfs_create_singular_buffer(rreq, folio);
if (ret < 0)
goto error_put;
@@ -883,7 +881,7 @@ int netfs_prefetch_for_write(struct file *file, struct folio *folio,
trace_netfs_read(rreq, start, flen, netfs_read_trace_prefetch_for_write);
/* Set up the output buffer */
- ret = netfs_create_singular_buffer(rreq, folio, NETFS_ROLLBUF_PAGECACHE_MARK);
+ ret = netfs_create_singular_buffer(rreq, folio);
if (ret < 0)
goto error_put;
diff --git a/fs/netfs/bvecq.c b/fs/netfs/bvecq.c
new file mode 100644
index 0000000000000..e78882b5672b7
--- /dev/null
+++ b/fs/netfs/bvecq.c
@@ -0,0 +1,346 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Buffering helpers for bvec queues
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#include <linux/bvecq.h>
+#include "internal.h"
+
+void bvecq_dump(const struct bvecq *bq)
+{
+ int b = 0;
+
+ for (; bq; bq = bvecq_next(bq), b++) {
+ int skipz = 0;
+
+ pr_notice("BQ[%u] %u/%u\n", b, bq->nr_slots, bq->max_slots);
+ for (int s = 0; s < bq->nr_slots; s++) {
+ const struct bio_vec *bv = &bq->bv[s];
+
+ if (!bv->bv_page && !bv->bv_len && skipz < 2) {
+ skipz = 1;
+ continue;
+ }
+ if (skipz == 1)
+ pr_notice("BQ[%u:00-%02u] ...\n", b, s - 1);
+ skipz = 2;
+ pr_notice("BQ[%u:%02u] %10lx %04x %04x %u\n",
+ b, s,
+ bv->bv_page ? page_to_pfn(bv->bv_page) : 0,
+ bv->bv_offset, bv->bv_len,
+ bv->bv_page ? page_count(bv->bv_page) : 0);
+ }
+ }
+}
+EXPORT_SYMBOL(bvecq_dump);
+
+/**
+ * bvecq_alloc_one - Allocate a single bvecq node with unpopulated slots
+ * @nr_slots: Number of slots to allocate
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a single bvecq node and initialise the header. The allocation is
+ * rounded up to the size of the smallest slab granule that will accommodate it
+ * and all the remaining space is set as an inline slot array with max_slots
+ * set to the number of slots. The slot array is not initialised.
+ *
+ * If @for_writeback is set, then the emergency mempool may be used for
+ * allocation. If it does allocate from that pool, the maximum number of slots
+ * available will be BVECQ_POOL_SLOTS which may be less than requested. Also,
+ * if @for_writeback is set, the function will not fail - though it may have to
+ * wait.
+ *
+ * Return: The node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback)
+{
+ struct bvecq *bq;
+ const size_t pool_size = struct_size_t(struct bvecq, __bv, BVECQ_POOL_SLOTS);
+ size_t size;
+ bool from_pool = false;
+
+ size = kmalloc_size_roundup(struct_size(bq, __bv, nr_slots));
+ gfp &= ~(GFP_ZONEMASK | __GFP_THISNODE);
+
+ if (for_writeback) {
+ if (size != pool_size) {
+ gfp_t gfp_temp = gfp;
+
+ gfp_temp |= __GFP_NOMEMALLOC | __GFP_NORETRY | __GFP_NOWARN;
+ gfp_temp &= ~(__GFP_DIRECT_RECLAIM | __GFP_IO);
+ bq = kmalloc(size, gfp_temp);
+ if (bq)
+ goto success;
+ }
+
+ bq = mempool_alloc(&netfs_bvecq_pool, gfp);
+ if (!bq)
+ return bq;
+ from_pool = true;
+ size = pool_size;
+ } else {
+ bq = kmalloc(size, gfp);
+ if (!bq)
+ return bq;
+ }
+
+success:
+ *bq = (struct bvecq) {
+ .ref = REFCOUNT_INIT(1),
+ .bv = bq->__bv,
+ .inline_bv = true,
+ .max_slots = (size - sizeof(*bq)) / sizeof(bq->__bv[0]),
+ .from_pool = from_pool,
+ };
+ netfs_stat(&netfs_n_bvecq);
+ return bq;
+}
+EXPORT_SYMBOL(bvecq_alloc_one);
+
+/**
+ * bvecq_alloc_chain - Allocate an unpopulated bvecq chain
+ * @nr_slots: Number of slots to allocate
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a chain of bvecq nodes providing at least the requested cumulative
+ * number of slots. Each node is a maximum of 4KiB in size.
+ *
+ * Return: The first node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp, bool for_writeback)
+{
+ struct bvecq *head = NULL, *tail = NULL;
+
+ _enter("%zu", nr_slots);
+
+ for (;;) {
+ struct bvecq *bq;
+
+ bq = bvecq_alloc_one(min(nr_slots, BVECQ_4KB_SLOTS), gfp, for_writeback);
+ if (!bq)
+ goto oom;
+
+ if (tail)
+ bvecq_append(tail, bq);
+ else
+ head = bq;
+ tail = bq;
+ if (tail->max_slots >= nr_slots)
+ break;
+ nr_slots -= tail->max_slots;
+ }
+
+ return head;
+oom:
+ bvecq_put(head);
+ return NULL;
+}
+EXPORT_SYMBOL(bvecq_alloc_chain);
+
+/**
+ * bvecq_alloc_buffer2 - Allocate a bvecq chain and populate with buffers
+ * @size: Target size of the buffer (can be 0 for an empty buffer)
+ * @pre_slots: Number of preamble slots to set aside
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Allocate a chain of bvecq nodes and populate the slots with sufficient pages
+ * to provide at least the requested amount of space, leaving the first
+ * @pre_slots slots unset. The pre-slots must all fit into the the first
+ * bvecq.
+ *
+ * The pages allocated may be compound pages larger than PAGE_SIZE and thus
+ * occupy fewer slots. The pages have their refcounts set to 1 and can be
+ * passed to MSG_SPLICE_PAGES.
+ *
+ * Return: The first node pointer or NULL on allocation failure.
+ */
+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp,
+ bool for_writeback)
+{
+ struct bvecq *head = NULL, *p = NULL;
+ size_t nr_per_bq = BVECQ_POOL_SLOTS;
+ size_t count = pre_slots + DIV_ROUND_UP(size, PAGE_SIZE);
+
+ _enter("%zx,%zx,%u", size, count, pre_slots);
+
+ if (WARN_ON_ONCE(pre_slots > nr_per_bq))
+ return NULL;
+
+ head = bvecq_alloc_chain(count, gfp, for_writeback);
+ if (!head)
+ return NULL;
+
+ p = head;
+ do {
+ struct page **pages;
+ size_t unused, want, got, slot;
+
+ if (!count)
+ break;
+ if (WARN_ON_ONCE(!p))
+ goto oom;
+
+ if (p->nr_slots == 0) {
+ /* Need to clear pre slots and pages[], so just clear all. */
+ memset(p->bv, 0, p->max_slots * sizeof(p->bv[0]));
+ p->mem_type = BVECQ_MEM_ALLOCED;
+ p->nr_slots = pre_slots;
+ count -= pre_slots;
+ pre_slots = 0;
+ if (!count)
+ break;
+ }
+
+ if (p->nr_slots >= p->max_slots) {
+ p = p->next;
+ continue;
+ }
+ unused = p->max_slots - p->nr_slots;
+
+ pages = (struct page **)&p->bv[p->max_slots];
+ pages -= unused;
+
+ want = min(count, unused);
+ got = alloc_pages_bulk(gfp, want, pages);
+ if (!got)
+ goto oom;
+
+ slot = p->nr_slots;
+ for (int i = 0; i < got; i++) {
+ set_page_count(pages[i], 1);
+ bvec_set_page(&p->bv[slot++], pages[i], PAGE_SIZE, 0);
+ }
+
+ bvecq_filled_to(p, slot);
+ count -= got;
+ } while (count > 0);
+
+ return head;
+oom:
+ bvecq_put(head);
+ return NULL;
+}
+EXPORT_SYMBOL(bvecq_alloc_buffer2);
+
+/*
+ * Free the page pointed to by a slot as necessary.
+ */
+static void bvecq_free_slot(struct bvecq *bq, unsigned int slot)
+{
+ struct page *page = bq->bv[slot].bv_page;
+
+ if (!page)
+ return;
+
+ switch (bq->mem_type) {
+ case BVECQ_MEM_EXTERNAL:
+ break;
+ case BVECQ_MEM_PAGECACHE:
+ put_page(page);
+ break;
+ case BVECQ_MEM_GUP:
+ unpin_user_page(page);
+ break;
+ case BVECQ_MEM_ALLOCED:
+ __free_pages(page, compound_order(page));
+ break;
+ default:
+ WARN_ON_ONCE(1);
+ break;
+ }
+}
+
+/**
+ * bvecq_put - Put a ref on a bvec queue
+ * @bq: The start of the folio queue to free
+ *
+ * Put the ref(s) on the nodes in a bvec queue, freeing up the node and the
+ * page fragments it points to as the refcounts become zero.
+ */
+void bvecq_put(struct bvecq *bq)
+{
+ struct bvecq *next;
+
+ for (; bq; bq = next) {
+ if (!refcount_dec_and_test(&bq->ref))
+ break;
+ for (int slot = 0; slot < bq->nr_slots; slot++)
+ bvecq_free_slot(bq, slot);
+ next = bq->next;
+ netfs_stat_d(&netfs_n_bvecq);
+ if (bq->from_pool)
+ mempool_free(bq, &netfs_bvecq_pool);
+ else
+ kfree(bq);
+ }
+}
+EXPORT_SYMBOL(bvecq_put);
+
+/**
+ * bvecq_expand_buffer - Allocate buffer space into a bvec queue
+ * @_buffer: Pointer to the bvecq chain to expand (may point to a NULL; updated).
+ * @_cur_size: Current size of the buffer (updated).
+ * @size: Target size of the buffer.
+ * @gfp: The allocation constraints.
+ *
+ * Append extra pages to a buffer to increase its capacity to the @size
+ * specified. If the current tail has space, but is not of the
+ * BVECQ_MEM_ALLOCED memory type, a separate bvecq will be allocated to hold
+ * the new memory.
+ */
+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp)
+{
+ struct bvecq *tail = *_buffer;
+
+ size = round_up(size, PAGE_SIZE);
+ if (tail)
+ while (tail->next)
+ tail = tail->next;
+
+ while (*_cur_size < size) {
+ struct page *page;
+ size_t need = size - *_cur_size;
+ gfp_t gfp_add;
+ int order = 0;
+
+ if (!tail || bvecq_is_full(tail) || tail->mem_type != BVECQ_MEM_ALLOCED) {
+ struct bvecq *p;
+
+ p = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, false);
+ if (!p)
+ return -ENOMEM;
+ if (tail)
+ bvecq_append(tail, p);
+ else
+ *_buffer = p;
+ tail = p;
+ p->mem_type = BVECQ_MEM_ALLOCED;
+ }
+
+ if (need > PAGE_SIZE)
+ order = umin(ilog2(need) - PAGE_SHIFT, MAX_PAGECACHE_ORDER);
+
+ gfp_add = 0;
+ if (order > 0)
+ gfp_add |= __GFP_NORETRY | __GFP_NOWARN;
+ page = alloc_pages(gfp | __GFP_COMP | gfp_add, order);
+ if (!page && order > 0) {
+ page = alloc_pages(gfp | __GFP_COMP, 0);
+ order = 0;
+ }
+ if (!page)
+ return -ENOMEM;
+
+ bvec_set_page(&tail->bv[tail->nr_slots], page, PAGE_SIZE << order, 0);
+ *_cur_size += PAGE_SIZE << order;
+ bvecq_filled_to(tail, tail->nr_slots + 1);
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(bvecq_expand_buffer);
diff --git a/fs/netfs/internal.h b/fs/netfs/internal.h
index b8591abc90a98..583ffa967d5a9 100644
--- a/fs/netfs/internal.h
+++ b/fs/netfs/internal.h
@@ -7,7 +7,6 @@
#include <linux/slab.h>
#include <linux/seq_file.h>
-#include <linux/folio_queue.h>
#include <linux/netfs.h>
#include <linux/fscache.h>
#include <linux/fscache-cache.h>
@@ -45,7 +44,7 @@ extern struct list_head netfs_io_requests;
extern spinlock_t netfs_proc_lock;
extern mempool_t netfs_request_pool;
extern mempool_t netfs_subrequest_pool;
-extern mempool_t netfs_folioq_pool;
+extern mempool_t netfs_bvecq_pool;
#ifdef CONFIG_PROC_FS
static inline void netfs_proc_add_rreq(struct netfs_io_request *rreq)
@@ -70,8 +69,6 @@ static inline void netfs_proc_del_rreq(struct netfs_io_request *rreq) {}
/*
* misc.c
*/
-struct folio_queue *netfs_buffer_make_space(struct netfs_io_request *rreq,
- enum netfs_folioq_trace trace);
void netfs_reset_iter(struct netfs_io_subrequest *subreq);
void netfs_wake_collector(struct netfs_io_request *rreq);
void netfs_subreq_clear_in_progress(struct netfs_io_subrequest *subreq);
@@ -202,7 +199,7 @@ extern atomic_t netfs_n_wh_retry_write_req;
extern atomic_t netfs_n_wh_retry_write_subreq;
extern atomic_t netfs_n_wb_lock_skip;
extern atomic_t netfs_n_wb_lock_wait;
-extern atomic_t netfs_n_folioq;
+extern atomic_t netfs_n_bvecq;
int netfs_stats_show(struct seq_file *m, void *v);
diff --git a/fs/netfs/iterator.c b/fs/netfs/iterator.c
index eb1efb17f53aa..31748526d5682 100644
--- a/fs/netfs/iterator.c
+++ b/fs/netfs/iterator.c
@@ -245,33 +245,34 @@ static size_t netfs_limit_xarray(const struct iov_iter *iter, size_t start_offse
}
/*
- * Select the span of a folio queue iterator we're going to use. Limit it by
- * both maximum size and maximum number of segments. Returns the size of the
- * span in bytes.
+ * Select the span of a bvecq iterator we're going to use. Limit it by both
+ * maximum size and maximum number of segments. Returns the size of the span
+ * in bytes.
*/
-static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offset,
- size_t max_size, size_t max_segs)
+static size_t netfs_limit_bvecq(const struct iov_iter *iter, size_t start_offset,
+ size_t max_size, size_t max_segs)
{
- const struct folio_queue *folioq = iter->folioq;
+ const struct bvecq *bq = iter->bvecq;
unsigned int nsegs = 0;
- unsigned int slot = iter->folioq_slot;
+ unsigned int slot = iter->bvecq_slot;
size_t span = 0, n = iter->count;
- if (WARN_ON(!iov_iter_is_folioq(iter)) ||
+ if (WARN_ON(!iov_iter_is_bvecq(iter)) ||
WARN_ON(start_offset > n) ||
n == 0)
return 0;
max_size = umin(max_size, n - start_offset);
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = bvecq_next(bq);
slot = 0;
}
start_offset += iter->iov_offset;
do {
- size_t flen = folioq_folio_size(folioq, slot);
+ size_t flen;
+ flen = bq->bv[slot].bv_len;
if (start_offset < flen) {
span += flen - start_offset;
nsegs++;
@@ -283,11 +284,11 @@ static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offse
break;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = bvecq_next(bq);
slot = 0;
}
- } while (folioq);
+ } while (bq);
return umin(span, max_size);
}
@@ -295,8 +296,8 @@ static size_t netfs_limit_folioq(const struct iov_iter *iter, size_t start_offse
size_t netfs_limit_iter(const struct iov_iter *iter, size_t start_offset,
size_t max_size, size_t max_segs)
{
- if (iov_iter_is_folioq(iter))
- return netfs_limit_folioq(iter, start_offset, max_size, max_segs);
+ if (iov_iter_is_bvecq(iter))
+ return netfs_limit_bvecq(iter, start_offset, max_size, max_segs);
if (iov_iter_is_bvec(iter))
return netfs_limit_bvec(iter, start_offset, max_size, max_segs);
if (iov_iter_is_xarray(iter))
diff --git a/fs/netfs/main.c b/fs/netfs/main.c
index 609e22e8f76a0..36a1d98e593a0 100644
--- a/fs/netfs/main.c
+++ b/fs/netfs/main.c
@@ -28,7 +28,7 @@ static struct kmem_cache *netfs_request_slab;
static struct kmem_cache *netfs_subrequest_slab;
mempool_t netfs_request_pool;
mempool_t netfs_subrequest_pool;
-mempool_t netfs_folioq_pool;
+mempool_t netfs_bvecq_pool;
#ifdef CONFIG_PROC_FS
LIST_HEAD(netfs_io_requests);
@@ -108,8 +108,9 @@ static int __init netfs_init(void)
{
int ret = -ENOMEM;
- if (mempool_init_kmalloc_pool(&netfs_folioq_pool, 100, sizeof(struct folio_queue)) < 0)
- goto error_folioq_pool;
+ if (mempool_init_kmalloc_pool(&netfs_bvecq_pool, 100,
+ struct_size_t(struct bvecq, __bv, BVECQ_POOL_SLOTS)) < 0)
+ goto error_bvecq_pool;
netfs_request_slab = kmem_cache_create("netfs_request",
sizeof(struct netfs_io_request), 0,
@@ -163,8 +164,8 @@ static int __init netfs_init(void)
error_reqpool:
kmem_cache_destroy(netfs_request_slab);
error_req:
- mempool_exit(&netfs_folioq_pool);
-error_folioq_pool:
+ mempool_exit(&netfs_bvecq_pool);
+error_bvecq_pool:
return ret;
}
fs_initcall(netfs_init);
@@ -177,6 +178,6 @@ static void __exit netfs_exit(void)
kmem_cache_destroy(netfs_subrequest_slab);
mempool_exit(&netfs_request_pool);
kmem_cache_destroy(netfs_request_slab);
- mempool_exit(&netfs_folioq_pool);
+ mempool_exit(&netfs_bvecq_pool);
}
module_exit(netfs_exit);
diff --git a/fs/netfs/misc.c b/fs/netfs/misc.c
index a3cd76d584b8b..40e0ff6491329 100644
--- a/fs/netfs/misc.c
+++ b/fs/netfs/misc.c
@@ -8,102 +8,6 @@
#include <linux/swap.h>
#include "internal.h"
-/**
- * netfs_alloc_folioq_buffer - Allocate buffer space into a folio queue
- * @mapping: Address space to set on the folio (or NULL).
- * @_buffer: Pointer to the folio queue to add to (may point to a NULL; updated).
- * @_cur_size: Current size of the buffer (updated).
- * @size: Target size of the buffer.
- * @gfp: The allocation constraints.
- */
-int netfs_alloc_folioq_buffer(struct address_space *mapping,
- struct folio_queue **_buffer,
- size_t *_cur_size, ssize_t size, gfp_t gfp)
-{
- struct folio_queue *tail = *_buffer, *p;
-
- size = round_up(size, PAGE_SIZE);
- if (*_cur_size >= size)
- return 0;
-
- if (tail)
- while (tail->next)
- tail = tail->next;
-
- do {
- struct folio *folio;
- int order = 0, slot;
-
- if (!tail || folioq_full(tail)) {
- p = netfs_folioq_alloc(0, GFP_NOFS, netfs_trace_folioq_alloc_buffer);
- if (!p)
- return -ENOMEM;
- if (tail) {
- tail->next = p;
- p->prev = tail;
- } else {
- *_buffer = p;
- }
- tail = p;
- }
-
- if (size - *_cur_size > PAGE_SIZE)
- order = umin(ilog2(size - *_cur_size) - PAGE_SHIFT,
- MAX_PAGECACHE_ORDER);
-
- folio = folio_alloc(gfp, order);
- if (!folio && order > 0)
- folio = folio_alloc(gfp, 0);
- if (!folio)
- return -ENOMEM;
-
- folio->mapping = mapping;
- folio->index = *_cur_size / PAGE_SIZE;
- trace_netfs_folio(folio, netfs_folio_trace_alloc_buffer);
- slot = folioq_append_mark(tail, folio);
- *_cur_size += folioq_folio_size(tail, slot);
- } while (*_cur_size < size);
-
- return 0;
-}
-EXPORT_SYMBOL(netfs_alloc_folioq_buffer);
-
-/**
- * netfs_free_folioq_buffer - Free a folio queue.
- * @fq: The start of the folio queue to free
- *
- * Free up a chain of folio_queues and, if marked, the marked folios they point
- * to.
- */
-void netfs_free_folioq_buffer(struct folio_queue *fq)
-{
- struct folio_queue *next;
- struct folio_batch fbatch;
-
- folio_batch_init(&fbatch);
-
- for (; fq; fq = next) {
- for (int slot = 0; slot < folioq_count(fq); slot++) {
- struct folio *folio = folioq_folio(fq, slot);
-
- if (!folio ||
- !folioq_is_marked(fq, slot))
- continue;
-
- trace_netfs_folio(folio, netfs_folio_trace_put);
- if (folio_batch_add(&fbatch, folio))
- folio_batch_release(&fbatch);
- }
-
- netfs_stat_d(&netfs_n_folioq);
- next = fq->next;
- kfree(fq);
- }
-
- folio_batch_release(&fbatch);
-}
-EXPORT_SYMBOL(netfs_free_folioq_buffer);
-
/*
* Reset the subrequest iterator to refer just to the region remaining to be
* read. The iterator may or may not have been advanced by socket ops or
diff --git a/fs/netfs/read_collect.c b/fs/netfs/read_collect.c
index 2625efd48a9b1..6bbaaea693543 100644
--- a/fs/netfs/read_collect.c
+++ b/fs/netfs/read_collect.c
@@ -63,11 +63,11 @@ void netfs_cancel_copy_to_cache(struct netfs_io_request *rreq, struct folio *fol
* dirty and let writeback handle it.
*/
static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
- struct folio_queue *folioq,
+ struct bvecq *bq,
int slot)
{
struct netfs_folio *finfo;
- struct folio *folio = folioq_folio(folioq, slot);
+ struct folio *folio = bvec_folio(&bq->bv[slot]);
if (unlikely(folio_pos(folio) < rreq->abandon_to)) {
trace_netfs_folio(folio, netfs_folio_trace_abandon);
@@ -98,7 +98,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
trace_netfs_folio(folio, netfs_folio_trace_read_done);
}
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
} else {
// TODO: Use of PG_private_2 is deprecated.
if (folio_test_private_2(folio))
@@ -114,7 +114,7 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
folio_unlock(folio);
}
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
}
/*
@@ -122,21 +122,21 @@ static void netfs_unlock_read_folio(struct netfs_io_request *rreq,
*/
void netfs_read_set_unlock_at(struct netfs_io_request *rreq)
{
- struct folio_queue *folioq = rreq->buffer.tail;
+ const struct bvecq *bq = rreq->buffer.tail;
unsigned int slot = rreq->buffer.first_tail_slot;
size_t cleaned_to = rreq->cleaned_to - rreq->start;
size_t progress_at = cleaned_to;
size_t minimum = 256 * 1024;
while (progress_at < rreq->len) {
- if (slot >= folioq_count(folioq)) {
- folioq = folioq->next;
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = bvecq_next(bq);
+ if (!bq)
break;
slot = 0;
}
- progress_at += folioq_folio_size(folioq, slot);
+ progress_at += bq->bv[slot].bv_len;
if (progress_at - cleaned_to >= minimum)
break;
slot++;
@@ -152,7 +152,7 @@ void netfs_read_set_unlock_at(struct netfs_io_request *rreq)
static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
unsigned int *notes)
{
- struct folio_queue *folioq = rreq->buffer.tail;
+ struct bvecq *bq = rreq->buffer.tail;
unsigned int slot = rreq->buffer.first_tail_slot;
uoff_t collected_to = rreq->collected_to;
@@ -161,9 +161,9 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
// TODO: Begin decryption
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&rreq->buffer);
- if (!folioq) {
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&rreq->buffer);
+ if (!bq) {
WRITE_ONCE(rreq->progress_at, rreq->len);
return;
}
@@ -182,13 +182,13 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
uoff_t fpos, fend;
size_t fsize;
- folio = folioq_folio(folioq, slot);
+ folio = bvec_folio(&bq->bv[slot]);
if (WARN_ONCE(!folio_test_locked(folio),
"R=%08x: folio %lx is not locked\n",
rreq->debug_id, folio->index))
trace_netfs_folio(folio, netfs_folio_trace_not_locked);
- fsize = folioq_folio_size(folioq, slot);
+ fsize = bq->bv[slot].bv_len;
fpos = folio_pos(folio);
fend = fpos + fsize;
@@ -198,29 +198,28 @@ static void netfs_read_unlock_folios(struct netfs_io_request *rreq,
if (collected_to < fend)
break;
- netfs_unlock_read_folio(rreq, folioq, slot);
+ netfs_unlock_read_folio(rreq, bq, slot);
WRITE_ONCE(rreq->cleaned_to, fpos + fsize);
*notes |= MADE_PROGRESS;
- /* Clean up the head folioq. If we clear an entire folioq, then
- * we can get rid of it provided it's not also the tail folioq
+ /* Clean up the head bq. If we clear an entire bq, then
+ * we can get rid of it provided it's not also the tail bq
* being filled by the issuer.
*/
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&rreq->buffer);
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&rreq->buffer);
+ if (!bq)
goto done;
slot = 0;
- trace_netfs_folioq(folioq, netfs_trace_folioq_read_progress);
}
if (fpos + fsize >= collected_to)
break;
}
- rreq->buffer.tail = folioq;
+ rreq->buffer.tail = bq;
done:
rreq->buffer.first_tail_slot = slot;
diff --git a/fs/netfs/read_pgpriv2.c b/fs/netfs/read_pgpriv2.c
index 5280b606fda41..16d623ca9df3e 100644
--- a/fs/netfs/read_pgpriv2.c
+++ b/fs/netfs/read_pgpriv2.c
@@ -53,7 +53,7 @@ static void netfs_pgpriv2_copy_folio(struct netfs_io_request *creq, struct folio
trace_netfs_folio(folio, netfs_folio_trace_store_copy);
/* Attach the folio to the rolling buffer. */
- if (rolling_buffer_append(&creq->buffer, folio, 0, creq->gfp) < 0) {
+ if (rolling_buffer_append(&creq->buffer, folio, creq->gfp) < 0) {
set_bit(NETFS_RREQ_CANCEL_CACHING, &creq->flags);
folio_end_private_2(folio);
return;
@@ -173,13 +173,13 @@ void netfs_pgpriv2_end_copy_to_cache(struct netfs_io_request *rreq)
*/
bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
{
- struct folio_queue *folioq = creq->buffer.tail;
+ struct bvecq *bq = creq->buffer.tail;
unsigned int slot = creq->buffer.first_tail_slot;
uoff_t collected_to = creq->collected_to;
bool made_progress = false;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&creq->buffer);
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&creq->buffer);
slot = 0;
}
@@ -188,7 +188,7 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
uoff_t fpos, fend;
size_t fsize, flen;
- folio = folioq_folio(folioq, slot);
+ folio = bvec_folio(&bq->bv[slot]);
if (WARN_ONCE(!folio_test_private_2(folio),
"R=%08x: folio %lx is not marked private_2\n",
creq->debug_id, folio->index))
@@ -211,15 +211,15 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
creq->cleaned_to = fpos + fsize;
made_progress = true;
- /* Clean up the head folioq. If we clear an entire folioq, then
- * we can get rid of it provided it's not also the tail folioq
+ /* Clean up the head bq. If we clear an entire bq, then
+ * we can get rid of it provided it's not also the tail bq
* being filled by the issuer.
*/
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&creq->buffer);
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&creq->buffer);
+ if (!bq)
goto done;
slot = 0;
}
@@ -228,7 +228,7 @@ bool netfs_pgpriv2_unlock_copied_folios(struct netfs_io_request *creq)
break;
}
- creq->buffer.tail = folioq;
+ creq->buffer.tail = bq;
done:
creq->buffer.first_tail_slot = slot;
return made_progress;
diff --git a/fs/netfs/read_retry.c b/fs/netfs/read_retry.c
index 5bd8dee5a8346..142c3fb8dab17 100644
--- a/fs/netfs/read_retry.c
+++ b/fs/netfs/read_retry.c
@@ -291,7 +291,7 @@ void netfs_retry_reads(struct netfs_io_request *rreq)
*/
void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
{
- struct folio_queue *p;
+ struct bvecq *p;
/* We have to wait for readahead refs to have been released before we
* can unlock any folios as the ref-dropper walks i_pages and the only
@@ -301,24 +301,25 @@ void netfs_unlock_abandoned_read_pages(struct netfs_io_request *rreq)
netfs_wait_for_put_ra_refs(rreq);
for (p = rreq->buffer.tail; p; p = p->next) {
- for (int slot = 0; slot < folioq_count(p); slot++) {
- struct folio *folio = folioq_folio(p, slot);
+ for (int slot = rreq->buffer.first_tail_slot;
+ bvecq_acquire_slot(p, slot);
+ slot++) {
+ struct folio *folio;
- if (!folio)
+ if (!p->bv[slot].bv_page)
continue;
+
+ folio = bvec_folio(&p->bv[slot]);
netfs_cancel_copy_to_cache(rreq, folio);
- if (!folioq_is_marked2(p, slot)) {
- if (folio == rreq->no_unlock_folio &&
- test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO,
- &rreq->flags)) {
- _debug("no unlock");
- } else {
- trace_netfs_folio(folio,
- netfs_folio_trace_abandon);
- folio_unlock(folio);
- }
+ if (folio == rreq->no_unlock_folio &&
+ test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, &rreq->flags)) {
+ _debug("no unlock");
+ } else {
+ trace_netfs_folio(folio, netfs_folio_trace_abandon);
+ folio_unlock(folio);
}
}
+ rreq->buffer.first_tail_slot = 0;
}
}
diff --git a/fs/netfs/rolling_buffer.c b/fs/netfs/rolling_buffer.c
index d30d5ef6d86ec..66ce9add40122 100644
--- a/fs/netfs/rolling_buffer.c
+++ b/fs/netfs/rolling_buffer.c
@@ -12,96 +12,51 @@
#include <linux/slab.h>
#include "internal.h"
-static atomic_t debug_ids;
-
-/**
- * netfs_folioq_alloc - Allocate a folio_queue struct
- * @rreq_id: Associated debugging ID for tracing purposes
- * @gfp: Allocation constraints
- * @trace: Trace tag to indicate the purpose of the allocation
- *
- * Allocate, initialise and account the folio_queue struct and log a trace line
- * to mark the allocation.
- */
-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,
- unsigned int /*enum netfs_folioq_trace*/ trace)
-{
- struct folio_queue *fq;
-
- if (gfp == GFP_KERNEL)
- fq = mempool_alloc_noreserve(&netfs_folioq_pool, gfp);
- else
- fq = mempool_alloc(&netfs_folioq_pool, gfp);
- if (fq) {
- netfs_stat(&netfs_n_folioq);
- folioq_init(fq, rreq_id);
- fq->debug_id = atomic_inc_return(&debug_ids);
- trace_netfs_folioq(fq, trace);
- }
- return fq;
-}
-EXPORT_SYMBOL(netfs_folioq_alloc);
-
-/**
- * netfs_folioq_free - Free a folio_queue struct
- * @folioq: The object to free
- * @trace: Trace tag to indicate which free
- *
- * Free and unaccount the folio_queue struct.
- */
-void netfs_folioq_free(struct folio_queue *folioq,
- unsigned int /*enum netfs_trace_folioq*/ trace)
-{
- trace_netfs_folioq(folioq, trace);
- netfs_stat_d(&netfs_n_folioq);
- mempool_free(folioq, &netfs_folioq_pool);
-}
-EXPORT_SYMBOL(netfs_folioq_free);
-
/*
* Initialise a rolling buffer. We allocate an empty folio queue struct to so
* that the pointers can be independently driven by the producer and the
* consumer.
*/
-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,
- unsigned int direction, gfp_t gfp)
+int rolling_buffer_init(struct rolling_buffer *roll, unsigned int direction,
+ gfp_t gfp, bool for_writeback)
{
- struct folio_queue *fq;
+ struct bvecq *bq;
+
+ roll->for_writeback = for_writeback;
- fq = netfs_folioq_alloc(rreq_id, gfp, netfs_trace_folioq_rollbuf_init);
- if (!fq)
+ bq = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, for_writeback);
+ if (!bq)
return -ENOMEM;
- roll->head = fq;
- roll->tail = fq;
- iov_iter_folio_queue(&roll->iter, direction, fq, 0, 0, 0);
+ roll->head = bq;
+ roll->tail = bq;
+ iov_iter_bvec_queue(&roll->iter, direction, bq, 0, 0, 0);
return 0;
}
/*
- * Add another folio_queue to a rolling buffer if there's no space left.
+ * Add another bvecq to a rolling buffer if there's no space left.
*/
int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)
{
- struct folio_queue *fq, *head = roll->head;
+ struct bvecq *bq, *head = roll->head;
- if (!folioq_full(head))
+ if (!bvecq_is_full(head))
return 0;
- fq = netfs_folioq_alloc(head->rreq_id, gfp, netfs_trace_folioq_make_space);
- if (!fq)
+ bq = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, roll->for_writeback);
+ if (!bq)
return -ENOMEM;
- fq->prev = head;
- roll->head = fq;
- if (folioq_full(head)) {
+ roll->head = bq;
+ if (bvecq_is_full(head)) {
/* Make sure we don't leave the master iterator pointing to a
* block that might get immediately consumed.
*/
- if (roll->iter.folioq == head &&
- roll->iter.folioq_slot == folioq_nr_slots(head)) {
- roll->iter.folioq = fq;
- roll->iter.folioq_slot = 0;
+ if (roll->iter.bvecq == head &&
+ roll->iter.bvecq_slot == head->nr_slots) {
+ roll->iter.bvecq = bq;
+ roll->iter.bvecq_slot = 0;
}
}
@@ -110,7 +65,7 @@ int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)
* [!] NOTE: After we set head->next, the consumer is at liberty to
* immediately delete the old head.
*/
- smp_store_release(&head->next, fq);
+ bvecq_append(head, bq);
return 0;
}
@@ -119,55 +74,59 @@ int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp)
*/
ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
struct readahead_control *ractl,
- unsigned int rreq_id, gfp_t gfp)
+ gfp_t gfp)
{
- struct folio_queue *fq;
- ssize_t loaded = 0;
+ struct bvecq *bq;
+ size_t loaded = 0;
while (ractl->_nr_pages - ractl->_batch_count > 0) {
+ struct page **pages;
unsigned int nr;
- /* Allocate a folioq to put some folios into and attach it to
+ /* Allocate a bvecq to put some folios into and attach it to
* the rolling buffer.
*/
- fq = netfs_folioq_alloc(rreq_id, gfp,
- netfs_trace_folioq_make_space);
- if (!fq)
+ bq = bvecq_alloc_one(BVECQ_POOL_SLOTS, gfp, false);
+ if (!bq)
goto nomem_unlock;
- fq->prev = roll->head;
+ bq->mem_type = BVECQ_MEM_EXTERNAL; /* Folio cleanup handled separately. */
+
if (!roll->tail)
- roll->tail = fq;
+ roll->tail = bq;
else
- roll->head->next = fq;
- roll->head = fq;
+ bvecq_append(roll->head, bq);
+ roll->head = bq;
- /* Get a batch of folios and note their orders. */
- nr = __readahead_batch(ractl, (struct page **)fq->vec.folios,
- folioq_nr_slots(fq));
+ /* Get a bunch of folios and note their sizes. */
+ pages = (struct page **)(bq->bv + bq->max_slots);
+ pages -= bq->max_slots;
+ nr = __readahead_batch(ractl, pages, bq->max_slots);
if (WARN_ON_ONCE(!nr))
break;
- fq->vec.nr = nr;
for (int slot = 0; slot < nr; slot++) {
- struct folio *folio = folioq_folio(fq, slot);
- unsigned int order;
+ struct folio *folio = page_folio(pages[slot]);
+ size_t len = folio_size(folio);
- order = folio_order(folio);
- fq->orders[slot] = order;
- loaded += PAGE_SIZE << order;
+ bvec_set_folio(&bq->bv[slot], folio, len, 0);
+ loaded += len;
trace_netfs_folio(folio, netfs_folio_trace_read);
}
+
+ bvecq_filled_to(bq, nr);
}
WRITE_ONCE(roll->iter.count, loaded);
- iov_iter_folio_queue(&roll->iter, ITER_DEST, roll->tail, 0, 0, loaded);
+ iov_iter_bvec_queue(&roll->iter, ITER_DEST, roll->tail, 0, 0, loaded);
return loaded;
nomem_unlock:
- for (fq = roll->tail; fq; fq = fq->next) {
- for (int slot = 0; slot < folioq_count(fq); slot++) {
- folio_unlock(fq->vec.folios[slot]);
- folioq_mark(fq, slot);
+ for (bq = roll->tail; bq; bq = bq->next) {
+ for (int slot = 0; slot < bq->nr_slots; slot++) {
+ struct folio *folio = bvec_folio(&bq->bv[slot]);
+
+ folio_unlock(folio);
+ folio_put(folio);
}
}
rolling_buffer_clear(roll);
@@ -180,7 +139,7 @@ ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
* Append a folio to the rolling buffer.
*/
ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
- unsigned int flags, gfp_t gfp)
+ gfp_t gfp)
{
ssize_t size = folio_size(folio);
int slot;
@@ -188,16 +147,11 @@ ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
if (rolling_buffer_make_space(roll, gfp) < 0)
return -ENOMEM;
- slot = folioq_append(roll->head, folio);
- if (flags & ROLLBUF_MARK_1)
- folioq_mark(roll->head, slot);
- if (flags & ROLLBUF_MARK_2)
- folioq_mark2(roll->head, slot);
+ slot = roll->head->nr_slots;
+ bvec_set_folio(&roll->head->bv[slot], folio, size, 0);
+ bvecq_filled_to(roll->head, slot + 1);
WRITE_ONCE(roll->iter.count, roll->iter.count + size);
-
- /* Store the counter after setting the slot. */
- smp_store_release(&roll->next_head_slot, slot);
return size;
}
@@ -206,44 +160,23 @@ ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
* don't return the last buffer to keep the pointers independent, but return
* NULL instead.
*/
-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll)
+struct bvecq *rolling_buffer_delete_spent(struct rolling_buffer *roll)
{
- struct folio_queue *spent = roll->tail, *next = READ_ONCE(spent->next);
+ struct bvecq *spent = roll->tail, *next = bvecq_next(spent);
if (!next)
return NULL;
next->prev = NULL;
- netfs_folioq_free(spent, netfs_trace_folioq_delete);
roll->tail = next;
+ spent->next = NULL;
+ bvecq_put(spent);
return next;
}
/*
- * Clear out a rolling queue. Folios that have mark 1 set are put.
+ * Clear out a rolling queue.
*/
void rolling_buffer_clear(struct rolling_buffer *roll)
{
- struct folio_batch fbatch;
- struct folio_queue *p;
-
- folio_batch_init(&fbatch);
-
- while ((p = roll->tail)) {
- roll->tail = p->next;
- for (int slot = 0; slot < folioq_count(p); slot++) {
- struct folio *folio = folioq_folio(p, slot);
-
- if (!folio)
- continue;
- if (folioq_is_marked(p, slot)) {
- trace_netfs_folio(folio, netfs_folio_trace_put);
- if (!folio_batch_add(&fbatch, folio))
- folio_batch_release(&fbatch);
- }
- }
-
- netfs_folioq_free(p, netfs_trace_folioq_clear);
- }
-
- folio_batch_release(&fbatch);
+ bvecq_put(roll->tail);
}
diff --git a/fs/netfs/stats.c b/fs/netfs/stats.c
index 9a607c4e62dda..0ba6ce9295b25 100644
--- a/fs/netfs/stats.c
+++ b/fs/netfs/stats.c
@@ -46,7 +46,7 @@ atomic_t netfs_n_wh_retry_write_req;
atomic_t netfs_n_wh_retry_write_subreq;
atomic_t netfs_n_wb_lock_skip;
atomic_t netfs_n_wb_lock_wait;
-atomic_t netfs_n_folioq;
+atomic_t netfs_n_bvecq;
int netfs_stats_show(struct seq_file *m, void *v)
{
@@ -88,10 +88,10 @@ int netfs_stats_show(struct seq_file *m, void *v)
atomic_read(&netfs_n_rh_retry_read_subreq),
atomic_read(&netfs_n_wh_retry_write_req),
atomic_read(&netfs_n_wh_retry_write_subreq));
- seq_printf(m, "Objs : rr=%u sr=%u foq=%u wsc=%u\n",
+ seq_printf(m, "Objs : rr=%u sr=%u bq=%u wsc=%u\n",
atomic_read(&netfs_n_rh_rreq),
atomic_read(&netfs_n_rh_sreq),
- atomic_read(&netfs_n_folioq),
+ atomic_read(&netfs_n_bvecq),
atomic_read(&netfs_n_wh_wstream_conflict));
seq_printf(m, "WbLock : skip=%u wait=%u\n",
atomic_read(&netfs_n_wb_lock_skip),
diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 6e8ea534230df..80e6a3194a609 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -114,11 +114,11 @@ int netfs_folio_written_back(struct folio *folio)
static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
unsigned int *notes)
{
- struct folio_queue *folioq = wreq->buffer.tail;
+ struct bvecq *bq = wreq->buffer.tail;
unsigned int slot = wreq->buffer.first_tail_slot;
uoff_t collected_to = wreq->collected_to;
- if (WARN_ON_ONCE(!folioq)) {
+ if (WARN_ON_ONCE(!bq)) {
pr_err("[!] Writeback unlock found empty rolling buffer!\n");
netfs_dump_request(wreq);
return;
@@ -130,9 +130,9 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
return;
}
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&wreq->buffer);
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&wreq->buffer);
+ if (!bq)
return;
slot = 0;
}
@@ -143,7 +143,7 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
uoff_t fpos, fend;
size_t fsize, flen;
- folio = folioq_folio(folioq, slot);
+ folio = bvec_folio(&bq->bv[slot]);
if (WARN_ONCE(!folio_test_writeback(folio),
"R=%08x: folio %lx is not under writeback\n",
wreq->debug_id, folio->index))
@@ -166,15 +166,15 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
wreq->cleaned_to = fpos + fsize;
*notes |= MADE_PROGRESS;
- /* Clean up the head folioq. If we clear an entire folioq, then
- * we can get rid of it provided it's not also the tail folioq
+ /* Clean up the head bq. If we clear an entire bq, then
+ * we can get rid of it provided it's not also the tail bq
* being filled by the issuer.
*/
- folioq_clear(folioq, slot);
+ bq->bv[slot].bv_page = NULL;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = rolling_buffer_delete_spent(&wreq->buffer);
- if (!folioq)
+ if (!bvecq_acquire_slot(bq, slot)) {
+ bq = rolling_buffer_delete_spent(&wreq->buffer);
+ if (!bq)
goto done;
slot = 0;
}
@@ -183,7 +183,7 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
break;
}
- wreq->buffer.tail = folioq;
+ wreq->buffer.tail = bq;
done:
wreq->buffer.first_tail_slot = slot;
}
diff --git a/fs/netfs/write_issue.c b/fs/netfs/write_issue.c
index 3989b4ec0c4b4..9de73c0edad2f 100644
--- a/fs/netfs/write_issue.c
+++ b/fs/netfs/write_issue.c
@@ -107,7 +107,9 @@ struct netfs_io_request *netfs_create_write_req(struct address_space *mapping,
ictx = netfs_inode(wreq->inode);
if (is_cacheable)
fscache_begin_write_operation(&wreq->cache_resources, netfs_i_cookie(ictx));
- if (rolling_buffer_init(&wreq->buffer, wreq->debug_id, ITER_SOURCE, wreq->gfp) < 0)
+ if (rolling_buffer_init(&wreq->buffer, ITER_SOURCE, wreq->gfp,
+ (origin == NETFS_WRITEBACK ||
+ origin == NETFS_WRITEBACK_SINGLE)) < 0)
goto nomem;
wreq->cleaned_to = wreq->start;
@@ -162,12 +164,12 @@ void netfs_prepare_write(struct netfs_io_request *wreq,
struct netfs_io_subrequest *subreq;
struct iov_iter *wreq_iter = &wreq->buffer.iter;
- /* Make sure we don't point the iterator at a used-up folio_queue
- * struct being used as a placeholder to prevent the queue from
- * collapsing. In such a case, extend the queue.
+ /* Make sure we don't point the iterator at a used-up bvecq struct
+ * being used as a placeholder to prevent the queue from collapsing.
+ * In such a case, extend the queue.
*/
- if (iov_iter_is_folioq(wreq_iter) &&
- wreq_iter->folioq_slot >= folioq_nr_slots(wreq_iter->folioq))
+ if (iov_iter_is_bvecq(wreq_iter) &&
+ !bvecq_acquire_slot(wreq_iter->bvecq, wreq_iter->bvecq_slot))
rolling_buffer_make_space(&wreq->buffer, wreq->gfp);
subreq = netfs_alloc_subrequest(wreq, stream->source);
@@ -450,7 +452,7 @@ static int netfs_write_folio(struct netfs_io_request *wreq,
}
/* Attach the folio to the rolling buffer. */
- rolling_buffer_append(&wreq->buffer, folio, 0, wreq->gfp);
+ rolling_buffer_append(&wreq->buffer, folio, wreq->gfp);
/* Move the submission point forward to allow for write-streaming data
* not starting at the front of the page. We don't do write-streaming
@@ -624,129 +626,11 @@ int netfs_writepages(struct address_space *mapping,
}
EXPORT_SYMBOL(netfs_writepages);
-/*
- * Write some of a pending folio data back to the server and/or the cache.
- */
-static int netfs_write_folio_single(struct netfs_io_request *wreq,
- struct folio *folio)
-{
- struct netfs_io_stream *upload = &wreq->io_streams[0];
- struct netfs_io_stream *cache = &wreq->io_streams[1];
- struct netfs_io_stream *stream;
- size_t iter_off = 0;
- size_t fsize = folio_size(folio), flen;
- uoff_t fpos = folio_pos(folio);
- ssize_t ret;
- bool to_eof = false;
- bool no_debug = false;
-
- _enter("");
-
- flen = folio_size(folio);
- if (flen > wreq->i_size - fpos) {
- flen = wreq->i_size - fpos;
- folio_zero_segment(folio, flen, fsize);
- to_eof = true;
- } else if (flen == wreq->i_size - fpos) {
- to_eof = true;
- }
-
- _debug("folio %zx/%zx", flen, fsize);
-
- if (!upload->avail && !cache->avail) {
- trace_netfs_folio(folio, netfs_folio_trace_cancel_store);
- return 0;
- }
-
- if (!upload->construct)
- trace_netfs_folio(folio, netfs_folio_trace_store);
- else
- trace_netfs_folio(folio, netfs_folio_trace_store_plus);
-
- /* Attach the folio to the rolling buffer. */
- folio_get(folio);
- ret = rolling_buffer_append(&wreq->buffer, folio, NETFS_ROLLBUF_PUT_MARK, wreq->gfp);
- if (ret < 0) {
- folio_put(folio);
- return ret;
- }
-
- /* Move the submission point forward to allow for write-streaming data
- * not starting at the front of the page. We don't do write-streaming
- * with the cache as the cache requires DIO alignment.
- *
- * Also skip uploading for data that's been read and just needs copying
- * to the cache.
- */
- for (int s = 0; s < NR_IO_STREAMS; s++) {
- stream = &wreq->io_streams[s];
- stream->submit_off = 0;
- stream->submit_len = flen;
- if (!stream->avail) {
- stream->submit_off = UINT_MAX;
- stream->submit_len = 0;
- }
- }
-
- /* Attach the folio to one or more subrequests. For a big folio, we
- * could end up with thousands of subrequests if the wsize is small -
- * but we might need to wait during the creation of subrequests for
- * network resources (eg. SMB credits).
- */
- for (;;) {
- ssize_t part;
- size_t lowest_off = ULONG_MAX;
- int choose_s = -1;
-
- /* Always add to the lowest-submitted stream first. */
- for (int s = 0; s < NR_IO_STREAMS; s++) {
- stream = &wreq->io_streams[s];
- if (stream->submit_len > 0 &&
- stream->submit_off < lowest_off) {
- lowest_off = stream->submit_off;
- choose_s = s;
- }
- }
-
- if (choose_s < 0)
- break;
- stream = &wreq->io_streams[choose_s];
-
- /* Advance the iterator(s). */
- if (stream->submit_off > iter_off) {
- rolling_buffer_advance(&wreq->buffer, stream->submit_off - iter_off);
- iter_off = stream->submit_off;
- }
-
- atomic64_set(&wreq->issued_to, fpos + stream->submit_off);
- stream->submit_extendable_to = fsize - stream->submit_off;
- part = netfs_advance_write(wreq, stream, fpos + stream->submit_off,
- stream->submit_len, to_eof);
- stream->submit_off += part;
- if (part > stream->submit_len)
- stream->submit_len = 0;
- else
- stream->submit_len -= part;
- if (part > 0)
- no_debug = true;
- }
-
- wreq->buffer.iter.iov_offset = 0;
- if (fsize > iter_off)
- rolling_buffer_advance(&wreq->buffer, fsize - iter_off);
- atomic64_set(&wreq->issued_to, fpos + fsize);
-
- if (!no_debug)
- kdebug("R=%x: No submit", wreq->debug_id);
- _leave(" = 0");
- return 0;
-}
-
/**
* netfs_writeback_single - Write back a monolithic payload
* @mapping: The mapping to write from
* @wbc: Hints from the VM
- * @iter: Data to write, must be ITER_FOLIOQ.
+ * @iter: Data to write.
*
* Write a monolithic, non-pagecache object back to the server and/or
* the cache.
@@ -760,13 +644,8 @@ int netfs_writeback_single(struct address_space *mapping,
{
struct netfs_io_request *wreq;
struct netfs_inode *ictx = netfs_inode(mapping->host);
- struct folio_queue *fq;
- size_t size = iov_iter_count(iter);
int ret;
- if (WARN_ON_ONCE(!iov_iter_is_folioq(iter)))
- return -EIO;
-
if (!netfs_wb_begin(ictx, wbc->sync_mode == WB_SYNC_NONE)) {
/* The VFS will have undirtied the inode. */
netfs_single_mark_inode_dirty(&ictx->inode);
@@ -779,36 +658,42 @@ int netfs_writeback_single(struct address_space *mapping,
goto couldnt_start;
}
+ wreq->buffer.iter = *iter;
+ wreq->len = iov_iter_count(iter);
+ wreq->submitted = wreq->len;
+
__set_bit(NETFS_RREQ_OFFLOAD_COLLECTION, &wreq->flags);
trace_netfs_write(wreq, netfs_write_trace_writeback_single);
netfs_stat(&netfs_n_wh_writepages);
- if (__test_and_set_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
+ if (test_bit(NETFS_RREQ_UPLOAD_TO_SERVER, &wreq->flags))
wreq->netfs_ops->begin_writeback(wreq);
- for (fq = (struct folio_queue *)iter->folioq; fq; fq = fq->next) {
- for (int slot = 0; slot < folioq_count(fq); slot++) {
- struct folio *folio = folioq_folio(fq, slot);
- size_t part = umin(folioq_folio_size(fq, slot), size);
+ for (int s = 0; s < NR_IO_STREAMS; s++) {
+ struct netfs_io_subrequest *subreq;
+ struct netfs_io_stream *stream = &wreq->io_streams[s];
- _debug("wbiter %lx %llx", folio->index, atomic64_read(&wreq->issued_to));
+ if (!stream->avail)
+ continue;
- ret = netfs_write_folio_single(wreq, folio);
- if (ret < 0)
- goto stop;
- size -= part;
- if (size <= 0)
- goto stop;
- }
+ netfs_prepare_write(wreq, stream, 0);
+
+ subreq = stream->construct;
+ subreq->len = wreq->len;
+ stream->submit_len = subreq->len;
+ stream->submit_extendable_to = round_up(wreq->len, PAGE_SIZE);
+
+ netfs_issue_write(wreq, stream);
}
-stop:
- for (int s = 0; s < NR_IO_STREAMS; s++)
- netfs_issue_write(wreq, &wreq->io_streams[s]);
netfs_all_subreqs_queued(wreq);
-
netfs_wake_collector(wreq);
+ /* TODO: Might want to be async here if WB_SYNC_NONE, but then need to
+ * wait before modifying.
+ */
+ ret = netfs_wait_for_write(wreq);
+
netfs_put_request(wreq, netfs_rreq_trace_put_return);
_leave(" = %d", ret);
return ret;
diff --git a/fs/smb/client/cifsglob.h b/fs/smb/client/cifsglob.h
index 79e4e84f8985c..c8b1c23ef4942 100644
--- a/fs/smb/client/cifsglob.h
+++ b/fs/smb/client/cifsglob.h
@@ -289,7 +289,7 @@ struct smb_rqst {
struct kvec *rq_iov; /* array of kvecs */
unsigned int rq_nvec; /* number of kvecs in array */
struct iov_iter rq_iter; /* Data iterator */
- struct folio_queue *rq_buffer; /* Buffer for encryption */
+ struct bvecq *rq_buffer; /* Buffer for encryption */
};
struct mid_q_entry;
diff --git a/fs/smb/client/smb2ops.c b/fs/smb/client/smb2ops.c
index aa142420dae2e..317d10b229666 100644
--- a/fs/smb/client/smb2ops.c
+++ b/fs/smb/client/smb2ops.c
@@ -13,7 +13,7 @@
#include <linux/sort.h>
#include <crypto/aead.h>
#include <linux/fiemap.h>
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
#include <uapi/linux/magic.h>
#include "cifsfs.h"
#include "cifsglob.h"
@@ -4830,19 +4830,18 @@ crypt_message(struct TCP_Server_Info *server, int num_rqst,
}
/*
- * Copy data from an iterator to the folios in a folio queue buffer.
+ * Copy data from an iterator to the pages in a bvec queue buffer.
*/
-static bool cifs_copy_iter_to_folioq(struct iov_iter *iter, size_t size,
- struct folio_queue *buffer)
+static bool cifs_copy_iter_to_bvecq(struct iov_iter *iter, size_t size,
+ struct bvecq *buffer)
{
for (; buffer; buffer = buffer->next) {
- for (int s = 0; s < folioq_count(buffer); s++) {
- struct folio *folio = folioq_folio(buffer, s);
- size_t part = folioq_folio_size(buffer, s);
+ for (int s = 0; s < bvecq_nr_slots_acquire(buffer); s++) {
+ struct bio_vec *bv = &buffer->bv[s];
+ size_t part = umin(bv->bv_len, size);
- part = umin(part, size);
-
- if (copy_folio_from_iter(folio, 0, part, iter) != part)
+ if (copy_page_from_iter(bv->bv_page, bv->bv_offset,
+ part, iter) != part)
return false;
size -= part;
}
@@ -4854,7 +4853,7 @@ void
smb3_free_compound_rqst(int num_rqst, struct smb_rqst *rqst)
{
for (int i = 0; i < num_rqst; i++)
- netfs_free_folioq_buffer(rqst[i].rq_buffer);
+ bvecq_put(rqst[i].rq_buffer);
}
/*
@@ -4881,7 +4880,7 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,
for (int i = 1; i < num_rqst; i++) {
struct smb_rqst *old = &old_rq[i - 1];
struct smb_rqst *new = &new_rq[i];
- struct folio_queue *buffer = NULL;
+ struct bvecq *buffer = NULL;
size_t size = iov_iter_count(&old->rq_iter);
orig_len += smb_rqst_len(server, old);
@@ -4889,17 +4888,16 @@ smb3_init_transform_rq(struct TCP_Server_Info *server, int num_rqst,
new->rq_nvec = old->rq_nvec;
if (size > 0) {
- size_t cur_size = 0;
- rc = netfs_alloc_folioq_buffer(NULL, &buffer, &cur_size,
- size, GFP_NOFS);
- new->rq_buffer = buffer;
- if (rc < 0)
+ rc = -ENOMEM;
+ buffer = bvecq_alloc_buffer(size, GFP_NOFS, true);
+ if (!buffer)
goto err_free;
- iov_iter_folio_queue(&new->rq_iter, ITER_SOURCE,
- buffer, 0, 0, size);
+ new->rq_buffer = buffer;
+ iov_iter_bvec_queue(&new->rq_iter, ITER_SOURCE,
+ buffer, 0, 0, size);
- if (!cifs_copy_iter_to_folioq(&old->rq_iter, size, buffer)) {
+ if (!cifs_copy_iter_to_bvecq(&old->rq_iter, size, buffer)) {
rc = smb_EIO1(smb_eio_trace_tx_copy_iter_to_buf, size);
goto err_free;
}
@@ -4989,22 +4987,20 @@ decrypt_raw_data(struct TCP_Server_Info *server, char *buf,
}
static int
-cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,
- size_t skip, struct iov_iter *iter)
+cifs_copy_bvecq_to_iter(struct bvecq *bq, size_t data_size,
+ size_t skip, struct iov_iter *iter)
{
- for (; folioq; folioq = folioq->next) {
- for (int s = 0; s < folioq_count(folioq); s++) {
- struct folio *folio;
- size_t fsize, n, len;
+ for (; bq; bq = bq->next) {
+ for (int s = 0; s < bvecq_nr_slots_acquire(bq); s++) {
+ struct bio_vec *bv = &bq->bv[s];
+ size_t n, len;
if (data_size == 0)
return 0;
- folio = folioq_folio(folioq, s);
- fsize = folio_size(folio);
- len = umin(fsize - skip, data_size);
+ len = umin(bv->bv_len - skip, data_size);
- n = copy_folio_to_iter(folio, skip, len, iter);
+ n = copy_page_to_iter(bv->bv_page, bv->bv_offset + skip, len, iter);
if (n != len) {
cifs_dbg(VFS, "%s: something went wrong\n", __func__);
return smb_EIO2(smb_eio_trace_rx_copy_to_iter,
@@ -5026,7 +5022,7 @@ cifs_copy_folioq_to_iter(struct folio_queue *folioq, size_t data_size,
static int
handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
- char *buf, unsigned int buf_len, struct folio_queue *buffer,
+ char *buf, unsigned int buf_len, struct bvecq *buffer,
unsigned int buffer_len, bool is_offloaded)
{
unsigned int data_offset;
@@ -5136,8 +5132,8 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
}
/* Copy the data to the output I/O iterator. */
- rdata->result = cifs_copy_folioq_to_iter(buffer, data_len,
- cur_off, &rdata->subreq.io_iter);
+ rdata->result = cifs_copy_bvecq_to_iter(buffer, data_len,
+ cur_off, &rdata->subreq.io_iter);
if (rdata->result != 0) {
if (is_offloaded)
mid->mid_state = MID_RESPONSE_MALFORMED;
@@ -5176,7 +5172,7 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
struct smb2_decrypt_work {
struct work_struct decrypt;
struct TCP_Server_Info *server;
- struct folio_queue *buffer;
+ struct bvecq *buffer;
char *buf;
unsigned int len;
};
@@ -5190,7 +5186,7 @@ static void smb2_decrypt_offload(struct work_struct *work)
struct mid_q_entry *mid;
struct iov_iter iter;
- iov_iter_folio_queue(&iter, ITER_DEST, dw->buffer, 0, 0, dw->len);
+ iov_iter_bvec_queue(&iter, ITER_DEST, dw->buffer, 0, 0, dw->len);
rc = decrypt_raw_data(dw->server, dw->buf, dw->server->vals->read_rsp_size,
&iter, true);
if (rc) {
@@ -5239,7 +5235,7 @@ static void smb2_decrypt_offload(struct work_struct *work)
}
free_pages:
- netfs_free_folioq_buffer(dw->buffer);
+ bvecq_put(dw->buffer);
cifs_small_buf_release(dw->buf);
kfree(dw);
}
@@ -5285,12 +5281,12 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,
dw->len = len;
len = round_up(dw->len, PAGE_SIZE);
- size_t cur_size = 0;
- rc = netfs_alloc_folioq_buffer(NULL, &dw->buffer, &cur_size, len, GFP_NOFS);
- if (rc < 0)
+ rc = -ENOMEM;
+ dw->buffer = bvecq_alloc_buffer(len, GFP_NOFS, false);
+ if (!dw->buffer)
goto discard_data;
- iov_iter_folio_queue(&iter, ITER_DEST, dw->buffer, 0, 0, len);
+ iov_iter_bvec_queue(&iter, ITER_DEST, dw->buffer, 0, 0, len);
/* Read the data into the buffer and clear excess bufferage. */
rc = cifs_read_iter_from_socket(server, &iter, dw->len);
@@ -5348,7 +5344,7 @@ receive_encrypted_read(struct TCP_Server_Info *server, struct mid_q_entry **mid,
}
free_pages:
- netfs_free_folioq_buffer(dw->buffer);
+ bvecq_put(dw->buffer);
free_dw:
kfree(dw);
return rc;
diff --git a/fs/smb/smbdirect/connection.c b/fs/smb/smbdirect/connection.c
index afd31fa12a368..aa52ecdbef2ba 100644
--- a/fs/smb/smbdirect/connection.c
+++ b/fs/smb/smbdirect/connection.c
@@ -5,7 +5,7 @@
*/
#include "internal.h"
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
struct smbdirect_map_sges {
struct ib_sge *sge;
@@ -2015,6 +2015,69 @@ static ssize_t smbdirect_map_sges_from_bvec(struct iov_iter *iter,
return ret;
}
+/*
+ * Extract memory fragments from a BVECQ-class iterator and add them to an RDMA
+ * list. The fragments are not pinned.
+ */
+static ssize_t smbdirect_map_sges_from_bvecq(struct iov_iter *iter,
+ struct smbdirect_map_sges *state,
+ ssize_t maxsize)
+{
+ const struct bvecq *bq = iter->bvecq, *next;
+ unsigned int slot = iter->bvecq_slot;
+ ssize_t extracted = 0;
+ size_t offset = iter->iov_offset;
+
+ maxsize = umin(maxsize, iov_iter_count(iter));
+
+ do {
+ struct bio_vec *bv;
+ size_t bsize;
+
+ while (slot >= bq->nr_slots) {
+ next = bvecq_next(bq);
+ if (!next) {
+ if (WARN_ON_ONCE(maxsize > 0))
+ return -EIO;
+ goto out;
+ }
+ bq = next;
+ slot = 0;
+ }
+
+ bv = &bq->bv[slot];
+ bsize = bv->bv_len;
+
+ if (offset < bsize) {
+ size_t part = umin(maxsize, bsize - offset);
+ bool ok;
+
+ ok = smbdirect_map_sges_single_page(state,
+ bv->bv_page,
+ bv->bv_offset + offset,
+ part);
+ if (!ok)
+ return -EIO;
+
+ offset += part;
+ extracted += part;
+ maxsize -= part;
+ }
+
+ if (offset >= bsize) {
+ offset = 0;
+ slot++;
+ }
+ } while (state->num_sge < state->max_sge && maxsize > 0);
+
+out:
+ iter->bvecq = bq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= extracted;
+ return extracted;
+}
+
/*
* Extract fragments from a KVEC-class iterator and add them to an ib_sge list.
* This can deal with vmalloc'd buffers as well as kmalloc'd or static buffers.
@@ -2076,70 +2139,6 @@ static ssize_t smbdirect_map_sges_from_kvec(struct iov_iter *iter,
return ret;
}
-/*
- * Extract folio fragments from a FOLIOQ-class iterator and add them to an
- * ib_sge list. The folios are not pinned.
- */
-static ssize_t smbdirect_map_sges_from_folioq(struct iov_iter *iter,
- struct smbdirect_map_sges *state,
- ssize_t maxsize)
-{
- const struct folio_queue *folioq = iter->folioq;
- unsigned int slot = iter->folioq_slot;
- ssize_t ret = 0;
- size_t offset = iter->iov_offset;
-
- if (WARN_ON_ONCE(!folioq))
- return -EIO;
-
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- if (WARN_ON_ONCE(!folioq))
- return -EIO;
- slot = 0;
- }
-
- do {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t fsize = folioq_folio_size(folioq, slot);
-
- if (offset < fsize) {
- size_t part = umin(maxsize, fsize - offset);
- bool ok;
-
- ok = smbdirect_map_sges_single_page(state,
- folio_page(folio, 0),
- offset,
- part);
- if (!ok)
- return -EIO;
-
- offset += part;
- ret += part;
- maxsize -= part;
- }
-
- if (offset >= fsize) {
- offset = 0;
- slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- if (!folioq->next) {
- WARN_ON_ONCE(ret < iter->count);
- break;
- }
- folioq = folioq->next;
- slot = 0;
- }
- }
- } while (state->num_sge < state->max_sge && maxsize > 0);
-
- iter->folioq = folioq;
- iter->folioq_slot = slot;
- iter->iov_offset = offset;
- iter->count -= ret;
- return ret;
-}
-
/*
* Extract page fragments from up to the given amount of the source iterator
* and build up an ib_sge list that refers to all of those bits. The ib_sge list
@@ -2164,12 +2163,12 @@ static ssize_t smbdirect_map_sges_from_iter(struct iov_iter *iter, size_t len,
case ITER_BVEC:
ret = smbdirect_map_sges_from_bvec(iter, state, len);
break;
+ case ITER_BVECQ:
+ ret = smbdirect_map_sges_from_bvecq(iter, state, len);
+ break;
case ITER_KVEC:
ret = smbdirect_map_sges_from_kvec(iter, state, len);
break;
- case ITER_FOLIOQ:
- ret = smbdirect_map_sges_from_folioq(iter, state, len);
- break;
default:
WARN_ONCE(1, "iov_iter_type[%u]\n", iov_iter_type(iter));
return -EIO;
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index fc566ee1c1ff1..df9ce860e7874 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -343,4 +343,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)
return page_to_phys(bvec->bv_page) + bvec->bv_offset;
}
+/**
+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ * @offset: Offset into bvec
+ *
+ * Map the page containing the byte at @offset into the kernel virtual address
+ * space. The caller is responsible for making sure this doesn't overrun.
+ *
+ * Call kunmap_local on the returned address to unmap.
+ */
+static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)
+{
+ offset += bvec->bv_offset;
+
+ return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
+ (offset & ~PAGE_MASK);
+}
+
#endif /* __LINUX_BVEC_H */
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
new file mode 100644
index 0000000000000..b984aaa449088
--- /dev/null
+++ b/include/linux/bvecq.h
@@ -0,0 +1,166 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Implementation of a segmented queue of bio_vec[].
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#ifndef _LINUX_BVECQ_H
+#define _LINUX_BVECQ_H
+
+#include <linux/bvec.h>
+
+/*
+ * The type of memory retention used by the elements in bvecq->bv[] and how to
+ * clean it up.
+ */
+enum bvecq_mem {
+ BVECQ_MEM_EXTERNAL, /* Externally retained memory - no freeing */
+ BVECQ_MEM_PAGECACHE, /* Ref'd pagecache pages - must put */
+ BVECQ_MEM_GUP, /* Pinned memory from get_user_pages() - unpin */
+ BVECQ_MEM_ALLOCED, /* Memory alloc'd by bvecq - can be freed/pooled */
+} __mode(byte);
+
+/*
+ * Segmented bio_vec queue.
+ *
+ * These can be linked together to form messages of indefinite length and
+ * iterated over with an ITER_BVECQ iterator. The list is non-circular; next
+ * and prev are NULL at the ends.
+ *
+ * The bv pointer points to the bio_vec array; this may be __bv if allocated
+ * together. The caller is responsible for determining whether or not this is
+ * the case as the array pointed to by bv may be follow on directly from the
+ * bvecq by accident of allocation (ie. ->bv == ->__bv is *not* sufficient to
+ * determine this).
+ */
+struct bvecq {
+ struct bvecq *next; /* Next bvec in the list or NULL */
+ struct bvecq *prev; /* Prev bvec in the list or NULL */
+ refcount_t ref;
+ u32 priv; /* Private data */
+ u16 nr_slots; /* Number of elements in bv[] used */
+ u16 max_slots; /* Number of elements allocated in bv[] */
+ enum bvecq_mem mem_type:3; /* What sort of memory and how to free it */
+ bool inline_bv:1; /* T if __bv[] is being used */
+ bool from_pool:1; /* T if bvecq from mempool */
+ struct bio_vec *bv; /* Pointer to array of page fragments */
+ struct bio_vec __bv[]; /* Default array (if ->inline_bv) */
+};
+
+/* Number of slots in a 512-byte mempool-backed bvecq. */
+#define BVECQ_POOL_SLOTS ((512 - sizeof(struct bvecq)) / sizeof(struct bio_vec))
+
+/* Number of slots in a 4K bvecq. */
+#define BVECQ_4KB_SLOTS ((4096 - sizeof(struct bvecq)) / sizeof(struct bio_vec))
+
+void bvecq_dump(const struct bvecq *bq);
+struct bvecq *bvecq_alloc_one(size_t nr_slots, gfp_t gfp, bool for_writeback);
+struct bvecq *bvecq_alloc_chain(size_t nr_slots, gfp_t gfp, bool for_writeback);
+struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp,
+ bool for_writeback);
+void bvecq_put(struct bvecq *bq);
+int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp);
+
+/**
+ * bvecq_alloc_buffer - Allocate a bvecq chain and populate with buffers
+ * @size: Target size of the buffer (can be 0 for an empty buffer)
+ * @gfp: The allocation constraints.
+ * @for_writeback: True if allocating for writeback
+ *
+ * Wrapper around %bvecq_alloc_buffer2().
+ */
+static inline struct bvecq *bvecq_alloc_buffer(size_t size, gfp_t gfp, bool for_writeback)
+{
+ return bvecq_alloc_buffer2(size, 0, gfp, for_writeback);
+}
+
+/**
+ * bvecq_get - Get a ref on a bvecq
+ * @bq: The bvecq to get a ref on
+ */
+static inline struct bvecq *bvecq_get(struct bvecq *bq)
+{
+ refcount_inc(&bq->ref);
+ return bq;
+}
+
+/**
+ * bvecq_is_full - Determine if a bvecq is full
+ * @bvecq: The object to query
+ *
+ * Return: true if full; false if not.
+ */
+static inline bool bvecq_is_full(const struct bvecq *bvecq)
+{
+ return bvecq->nr_slots >= bvecq->max_slots;
+}
+
+/**
+ * bvecq_filled_to - Release filled slots with release barrier
+ * @bvecq: The object modified
+ * @to: The latest slot filled + 1
+ */
+static inline void bvecq_filled_to(struct bvecq *bvecq, unsigned int to)
+{
+ /* Set the slot counter after filling the slot */
+ smp_store_release(&bvecq->nr_slots, to);
+}
+
+/**
+ * bvecq_nr_slots_acquire - Get the number of filled slots with acquire barrier
+ * @bvecq: The object to query
+ *
+ * Return: The number of filled slots
+ */
+static inline unsigned int bvecq_nr_slots_acquire(const struct bvecq *bvecq)
+{
+ /* Read the slot counter before looking at the slot */
+ return smp_load_acquire(&bvecq->nr_slots);
+}
+
+/**
+ * bvecq_acquire_slot - Determine if a slot is valid with acquire barrier
+ * @bvecq: The object to query
+ * @slot: The next slot
+ *
+ * Return: true if valid; false if might not be valid
+ */
+static inline bool bvecq_acquire_slot(const struct bvecq *bvecq, unsigned int slot)
+{
+ /* Read the slot counter before looking at the slot */
+ return slot < bvecq_nr_slots_acquire(bvecq);
+}
+
+/**
+ * bvecq_append - Get the next bvecq with appropriate barrier
+ * @to: The bvecq to append to
+ * @add: The bvecq to append
+ *
+ * Attach a new bvecq to a chain using an appropriate barrier to protect the
+ * write.
+ *
+ * [!] Note that this function transfers the caller's ref to the chain.
+ */
+static inline void bvecq_append(struct bvecq *to, struct bvecq *add)
+{
+ add->prev = to;
+
+ /* Make sure the initialisation is stored before the next pointer. */
+ smp_store_release(&to->next, add);
+}
+
+/**
+ * bvecq_next - Get the next bvecq with appropriate barrier
+ * @bq: The bvecq to start from
+ *
+ * Return the next bvecq in a chain, using an appropriate barrier to protect
+ * the access.
+ */
+static inline struct bvecq *bvecq_next(const struct bvecq *bq)
+{
+ /* Read the contents of the next node after the pointer to it. */
+ return smp_load_acquire(&bq->next);
+}
+
+#endif /* _LINUX_BVECQ_H */
diff --git a/include/linux/folio_queue.h b/include/linux/folio_queue.h
deleted file mode 100644
index f6d5f1f127c9f..0000000000000
--- a/include/linux/folio_queue.h
+++ /dev/null
@@ -1,282 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-or-later */
-/* Queue of folios definitions
- *
- * Copyright (C) 2024 Red Hat, Inc. All Rights Reserved.
- * Written by David Howells (dhowells@redhat.com)
- *
- * See:
- *
- * Documentation/core-api/folio_queue.rst
- *
- * for a description of the API.
- */
-
-#ifndef _LINUX_FOLIO_QUEUE_H
-#define _LINUX_FOLIO_QUEUE_H
-
-#include <linux/folio_batch.h>
-#include <linux/mm.h>
-
-/*
- * Segment in a queue of running buffers. Each segment can hold a number of
- * folios and a portion of the queue can be referenced with the ITER_FOLIOQ
- * iterator. The possibility exists of inserting non-folio elements into the
- * queue (such as gaps).
- *
- * Explicit prev and next pointers are used instead of a list_head to make it
- * easier to add segments to tail and remove them from the head without the
- * need for a lock.
- */
-struct folio_queue {
- struct folio_batch vec; /* Folios in the queue segment */
- u8 orders[FOLIO_BATCH_SIZE]; /* Order of each folio */
- struct folio_queue *next; /* Next queue segment or NULL */
- struct folio_queue *prev; /* Previous queue segment of NULL */
- unsigned long marks; /* 1-bit mark per folio */
- unsigned long marks2; /* Second 1-bit mark per folio */
-#if FOLIO_BATCH_SIZE > BITS_PER_LONG
-#error marks is not big enough
-#endif
- unsigned int rreq_id;
- unsigned int debug_id;
-};
-
-/**
- * folioq_init - Initialise a folio queue segment
- * @folioq: The segment to initialise
- * @rreq_id: The request identifier to use in tracelines.
- *
- * Initialise a folio queue segment and set an identifier to be used in traces.
- *
- * Note that the folio pointers are left uninitialised.
- */
-static inline void folioq_init(struct folio_queue *folioq, unsigned int rreq_id)
-{
- folio_batch_init(&folioq->vec);
- folioq->next = NULL;
- folioq->prev = NULL;
- folioq->marks = 0;
- folioq->marks2 = 0;
- folioq->rreq_id = rreq_id;
- folioq->debug_id = 0;
-}
-
-/**
- * folioq_nr_slots: Query the capacity of a folio queue segment
- * @folioq: The segment to query
- *
- * Query the number of folios that a particular folio queue segment might hold.
- * [!] NOTE: This must not be assumed to be the same for every segment!
- */
-static inline unsigned int folioq_nr_slots(const struct folio_queue *folioq)
-{
- return FOLIO_BATCH_SIZE;
-}
-
-/**
- * folioq_count: Query the occupancy of a folio queue segment
- * @folioq: The segment to query
- *
- * Query the number of folios that have been added to a folio queue segment.
- * Note that this is not decreased as folios are removed from a segment.
- */
-static inline unsigned int folioq_count(struct folio_queue *folioq)
-{
- return folio_batch_count(&folioq->vec);
-}
-
-/**
- * folioq_full: Query if a folio queue segment is full
- * @folioq: The segment to query
- *
- * Query if a folio queue segment is fully occupied. Note that this does not
- * change if folios are removed from a segment.
- */
-static inline bool folioq_full(struct folio_queue *folioq)
-{
- //return !folio_batch_space(&folioq->vec);
- return folioq_count(folioq) >= folioq_nr_slots(folioq);
-}
-
-/**
- * folioq_is_marked: Check first folio mark in a folio queue segment
- * @folioq: The segment to query
- * @slot: The slot number of the folio to query
- *
- * Determine if the first mark is set for the folio in the specified slot in a
- * folio queue segment.
- */
-static inline bool folioq_is_marked(const struct folio_queue *folioq, unsigned int slot)
-{
- return test_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_mark: Set the first mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Set the first mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_mark(struct folio_queue *folioq, unsigned int slot)
-{
- set_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_unmark: Clear the first mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Clear the first mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_unmark(struct folio_queue *folioq, unsigned int slot)
-{
- clear_bit(slot, &folioq->marks);
-}
-
-/**
- * folioq_is_marked2: Check second folio mark in a folio queue segment
- * @folioq: The segment to query
- * @slot: The slot number of the folio to query
- *
- * Determine if the second mark is set for the folio in the specified slot in a
- * folio queue segment.
- */
-static inline bool folioq_is_marked2(const struct folio_queue *folioq, unsigned int slot)
-{
- return test_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_mark2: Set the second mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Set the second mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_mark2(struct folio_queue *folioq, unsigned int slot)
-{
- set_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_unmark2: Clear the second mark on a folio in a folio queue segment
- * @folioq: The segment to modify
- * @slot: The slot number of the folio to modify
- *
- * Clear the second mark for the folio in the specified slot in a folio queue
- * segment.
- */
-static inline void folioq_unmark2(struct folio_queue *folioq, unsigned int slot)
-{
- clear_bit(slot, &folioq->marks2);
-}
-
-/**
- * folioq_append: Add a folio to a folio queue segment
- * @folioq: The segment to add to
- * @folio: The folio to add
- *
- * Add a folio to the tail of the sequence in a folio queue segment, increasing
- * the occupancy count and returning the slot number for the folio just added.
- * The folio size is extracted and stored in the queue and the marks are left
- * unmodified.
- *
- * Note that it's left up to the caller to check that the segment capacity will
- * not be exceeded and to extend the queue.
- */
-static inline unsigned int folioq_append(struct folio_queue *folioq, struct folio *folio)
-{
- unsigned int slot = folioq->vec.nr++;
-
- folioq->vec.folios[slot] = folio;
- folioq->orders[slot] = folio_order(folio);
- return slot;
-}
-
-/**
- * folioq_append_mark: Add a folio to a folio queue segment
- * @folioq: The segment to add to
- * @folio: The folio to add
- *
- * Add a folio to the tail of the sequence in a folio queue segment, increasing
- * the occupancy count and returning the slot number for the folio just added.
- * The folio size is extracted and stored in the queue, the first mark is set
- * and and the second and third marks are left unmodified.
- *
- * Note that it's left up to the caller to check that the segment capacity will
- * not be exceeded and to extend the queue.
- */
-static inline unsigned int folioq_append_mark(struct folio_queue *folioq, struct folio *folio)
-{
- unsigned int slot = folioq->vec.nr++;
-
- folioq->vec.folios[slot] = folio;
- folioq->orders[slot] = folio_order(folio);
- folioq_mark(folioq, slot);
- return slot;
-}
-
-/**
- * folioq_folio: Get a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the folio in the specified slot from a folio queue segment. Note
- * that no bounds check is made and if the slot hasn't been added into yet, the
- * pointer will be undefined. If the slot has been cleared, NULL will be
- * returned.
- */
-static inline struct folio *folioq_folio(const struct folio_queue *folioq, unsigned int slot)
-{
- return folioq->vec.folios[slot];
-}
-
-/**
- * folioq_folio_order: Get the order of a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the order of the folio in the specified slot from a folio queue
- * segment. Note that no bounds check is made and if the slot hasn't been
- * added into yet, the order returned will be 0.
- */
-static inline unsigned int folioq_folio_order(const struct folio_queue *folioq, unsigned int slot)
-{
- return folioq->orders[slot];
-}
-
-/**
- * folioq_folio_size: Get the size of a folio from a folio queue segment
- * @folioq: The segment to access
- * @slot: The folio slot to access
- *
- * Retrieve the size of the folio in the specified slot from a folio queue
- * segment. Note that no bounds check is made and if the slot hasn't been
- * added into yet, the size returned will be PAGE_SIZE.
- */
-static inline size_t folioq_folio_size(const struct folio_queue *folioq, unsigned int slot)
-{
- return PAGE_SIZE << folioq_folio_order(folioq, slot);
-}
-
-/**
- * folioq_clear: Clear a folio from a folio queue segment
- * @folioq: The segment to clear
- * @slot: The folio slot to clear
- *
- * Clear a folio from a sequence in a folio queue segment and clear its marks.
- * The occupancy count is left unchanged.
- */
-static inline void folioq_clear(struct folio_queue *folioq, unsigned int slot)
-{
- folioq->vec.folios[slot] = NULL;
- folioq_unmark(folioq, slot);
- folioq_unmark2(folioq, slot);
-}
-
-#endif /* _LINUX_FOLIO_QUEUE_H */
diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h
index f9a17fbbd3980..39ffd5878f505 100644
--- a/include/linux/iov_iter.h
+++ b/include/linux/iov_iter.h
@@ -10,7 +10,7 @@
#include <linux/uio.h>
#include <linux/bvec.h>
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
void *priv, void *priv2);
@@ -142,56 +142,65 @@ size_t iterate_bvec(struct iov_iter *iter, size_t len, void *priv, void *priv2,
}
/*
- * Handle ITER_FOLIOQ.
+ * Handle ITER_BVECQ.
*/
static __always_inline
-size_t iterate_folioq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
- iov_step_f step)
+size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
+ iov_step_f step)
{
- const struct folio_queue *folioq = iter->folioq;
- unsigned int slot = iter->folioq_slot;
+ const struct bvecq *bq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
size_t progress = 0, skip = iter->iov_offset;
- if (slot == folioq_nr_slots(folioq)) {
- /* The iterator may have been extended. */
- folioq = folioq->next;
- slot = 0;
- }
-
do {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t part, remain = 0, consumed;
- size_t fsize;
+ const struct bio_vec *bvec;
+ struct page *page;
+ size_t poff, plen;
void *base;
- if (!folio)
- break;
+ if (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
+ slot = 0;
+ continue;
+ }
+
+ bvec = &bq->bv[slot];
+ /*
+ * The caller must ensure that a slot with bv_len>0 has a valid
+ * bv_page.
+ */
+ page = bvec->bv_page + (bvec->bv_offset + skip) / PAGE_SIZE;
+ poff = (bvec->bv_offset + skip) % PAGE_SIZE;
+ plen = min(bvec->bv_len - skip, len);
+
+ while (plen > 0) {
+ size_t part, remain, consumed;
- fsize = folioq_folio_size(folioq, slot);
- if (skip < fsize) {
- base = kmap_local_folio(folio, skip);
- part = umin(len, PAGE_SIZE - skip % PAGE_SIZE);
+ part = min(plen, PAGE_SIZE - poff);
+ base = kmap_local_page(page) + poff;
remain = step(base, progress, part, priv, priv2);
kunmap_local(base);
+
consumed = part - remain;
- len -= consumed;
progress += consumed;
skip += consumed;
+ len -= consumed;
+ if (!len || remain)
+ goto stop;
+ page++;
+ poff = 0;
+ plen -= consumed;
}
- if (skip >= fsize) {
- skip = 0;
- slot++;
- if (slot == folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
- slot = 0;
- }
- }
- if (remain)
- break;
+
+ skip = 0;
+ slot++;
} while (len);
- iter->folioq_slot = slot;
- iter->folioq = folioq;
+stop:
+ iter->bvecq_slot = slot;
+ iter->bvecq = bq;
iter->iov_offset = skip;
iter->count -= progress;
return progress;
@@ -306,8 +315,8 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
- if (iov_iter_is_folioq(iter))
- return iterate_folioq(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
return iterate_xarray(iter, len, priv, priv2, step);
return iterate_discard(iter, len, priv, priv2, step);
@@ -342,7 +351,7 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
* buffer is presented in segments, which for kernel iteration are broken up by
* physical pages and mapped, with the mapped address being presented.
*
- * [!] Note This will only handle BVEC, KVEC, FOLIOQ, XARRAY and DISCARD-type
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, XARRAY and DISCARD-type
* iterators; it will not handle UBUF or IOVEC-type iterators.
*
* A step functions, @step, must be provided, one for handling mapped kernel
@@ -370,8 +379,8 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
- if (iov_iter_is_folioq(iter))
- return iterate_folioq(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
return iterate_xarray(iter, len, priv, priv2, step);
return iterate_discard(iter, len, priv, priv2, step);
diff --git a/include/linux/netfs.h b/include/linux/netfs.h
index 67e010b6994b3..31a80f4377d75 100644
--- a/include/linux/netfs.h
+++ b/include/linux/netfs.h
@@ -17,13 +17,13 @@
#include <linux/workqueue.h>
#include <linux/fs.h>
#include <linux/pagemap.h>
+#include <linux/bvecq.h>
#include <linux/uio.h>
#include <linux/rolling_buffer.h>
enum netfs_sreq_ref_trace;
typedef struct mempool mempool_t;
struct fscache_occupancy;
-struct folio_queue;
/**
* folio_start_private_2 - Start an fscache write on a folio. [DEPRECATED]
@@ -469,18 +469,6 @@ void netfs_end_io_write(struct inode *inode);
int netfs_start_io_direct(struct inode *inode);
void netfs_end_io_direct(struct inode *inode);
-/* Miscellaneous APIs. */
-struct folio_queue *netfs_folioq_alloc(unsigned int rreq_id, gfp_t gfp,
- unsigned int trace /*enum netfs_folioq_trace*/);
-void netfs_folioq_free(struct folio_queue *folioq,
- unsigned int trace /*enum netfs_trace_folioq*/);
-
-/* Buffer wrangling helpers API. */
-int netfs_alloc_folioq_buffer(struct address_space *mapping,
- struct folio_queue **_buffer,
- size_t *_cur_size, ssize_t size, gfp_t gfp);
-void netfs_free_folioq_buffer(struct folio_queue *fq);
-
/* Writeback exclusion API. */
bool netfs_wb_begin(struct netfs_inode *ictx, bool nowait);
void netfs_wb_end(struct netfs_inode *ictx);
diff --git a/include/linux/rolling_buffer.h b/include/linux/rolling_buffer.h
index a97f7cfaacaad..5c0bc4221f010 100644
--- a/include/linux/rolling_buffer.h
+++ b/include/linux/rolling_buffer.h
@@ -8,49 +8,35 @@
#ifndef _ROLLING_BUFFER_H
#define _ROLLING_BUFFER_H
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
#include <linux/uio.h>
/*
- * Rolling buffer. Whilst the buffer is live and in use, folios and folio
- * queue segments can be added to one end by one thread and removed from the
- * other end by another thread. The buffer isn't allowed to be empty; it must
- * always have at least one folio_queue in it so that neither side has to
- * modify both queue pointers.
+ * Rolling buffer. Whilst the buffer is live and in use, folios and bvecq
+ * segments can be added to one end by one thread and removed from the other
+ * end by another thread. The buffer isn't allowed to be empty; it must always
+ * have at least one bvecq in it so that neither side has to modify both queue
+ * pointers.
*
* The iterator in the buffer is extended as buffers are inserted. It can be
* snapshotted to use a segment of the buffer.
*/
struct rolling_buffer {
- struct folio_queue *head; /* Producer's insertion point */
- struct folio_queue *tail; /* Consumer's removal point */
+ struct bvecq *head; /* Producer's insertion point */
+ struct bvecq *tail; /* Consumer's removal point */
struct iov_iter iter; /* Iterator tracking what's left in the buffer */
- u8 next_head_slot; /* Next slot in ->head */
u8 first_tail_slot; /* First slot in ->tail */
+ bool for_writeback; /* T if being used for writeback */
};
-/*
- * Snapshot of a rolling buffer.
- */
-struct rolling_buffer_snapshot {
- struct folio_queue *curr_folioq; /* Queue segment in which current folio resides */
- unsigned char curr_slot; /* Folio currently being read */
- unsigned char curr_order; /* Order of folio */
-};
-
-/* Marks to store per-folio in the internal folio_queue structs. */
-#define ROLLBUF_MARK_1 BIT(0)
-#define ROLLBUF_MARK_2 BIT(1)
-
-int rolling_buffer_init(struct rolling_buffer *roll, unsigned int rreq_id,
- unsigned int direction, gfp_t gfp);
+int rolling_buffer_init(struct rolling_buffer *roll, unsigned int direction,
+ gfp_t gfp, bool for_writeback);
int rolling_buffer_make_space(struct rolling_buffer *roll, gfp_t gfp);
ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,
struct readahead_control *ractl,
- unsigned int rreq_id, gfp_t gfp);
-ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio,
- unsigned int flags, gfp_t gfp);
-struct folio_queue *rolling_buffer_delete_spent(struct rolling_buffer *roll);
+ gfp_t gfp);
+ssize_t rolling_buffer_append(struct rolling_buffer *roll, struct folio *folio, gfp_t gfp);
+struct bvecq *rolling_buffer_delete_spent(struct rolling_buffer *roll);
void rolling_buffer_clear(struct rolling_buffer *roll);
static inline void rolling_buffer_advance(struct rolling_buffer *roll, size_t amount)
diff --git a/include/linux/uio.h b/include/linux/uio.h
index fe2e985d74d24..8d82a8258ef11 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -11,7 +11,6 @@
#include <uapi/linux/uio.h>
struct page;
-struct folio_queue;
typedef unsigned int __bitwise iov_iter_extraction_t;
@@ -26,7 +25,7 @@ enum iter_type {
ITER_IOVEC,
ITER_BVEC,
ITER_KVEC,
- ITER_FOLIOQ,
+ ITER_BVECQ,
ITER_XARRAY,
ITER_DISCARD,
};
@@ -68,7 +67,7 @@ struct iov_iter {
const struct iovec *__iov;
const struct kvec *kvec;
const struct bio_vec *bvec;
- const struct folio_queue *folioq;
+ const struct bvecq *bvecq;
struct xarray *xarray;
void __user *ubuf;
};
@@ -77,7 +76,7 @@ struct iov_iter {
};
union {
unsigned long nr_segs;
- u8 folioq_slot;
+ u16 bvecq_slot;
loff_t xarray_start;
};
};
@@ -145,9 +144,9 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
return iov_iter_type(i) == ITER_DISCARD;
}
-static inline bool iov_iter_is_folioq(const struct iov_iter *i)
+static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
{
- return iov_iter_type(i) == ITER_FOLIOQ;
+ return iov_iter_type(i) == ITER_BVECQ;
}
static inline bool iov_iter_is_xarray(const struct iov_iter *i)
@@ -295,9 +294,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec
void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec,
unsigned long nr_segs, size_t count);
void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
- const struct folio_queue *folioq,
- unsigned int first_slot, unsigned int offset, size_t count);
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq,
+ unsigned int first_slot, unsigned int offset, size_t count);
void iov_iter_xarray(struct iov_iter *i, unsigned int direction, struct xarray *xarray,
loff_t start, size_t count);
ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,
diff --git a/include/trace/events/netfs.h b/include/trace/events/netfs.h
index bf1e1f185b05c..1f92a653d3bfa 100644
--- a/include/trace/events/netfs.h
+++ b/include/trace/events/netfs.h
@@ -244,14 +244,6 @@
EM(netfs_trace_donate_to_next, "to-next") \
E_(netfs_trace_donate_to_deferred_next, "defer-next")
-#define netfs_folioq_traces \
- EM(netfs_trace_folioq_alloc_buffer, "alloc-buf") \
- EM(netfs_trace_folioq_clear, "clear") \
- EM(netfs_trace_folioq_delete, "delete") \
- EM(netfs_trace_folioq_make_space, "make-space") \
- EM(netfs_trace_folioq_rollbuf_init, "roll-init") \
- E_(netfs_trace_folioq_read_progress, "r-progress")
-
#ifndef __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
#define __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
@@ -270,7 +262,6 @@ enum netfs_sreq_ref_trace { netfs_sreq_ref_traces } __mode(byte);
enum netfs_folio_trace { netfs_folio_traces } __mode(byte);
enum netfs_collect_contig_trace { netfs_collect_contig_traces } __mode(byte);
enum netfs_donate_trace { netfs_donate_traces } __mode(byte);
-enum netfs_folioq_trace { netfs_folioq_traces } __mode(byte);
#endif
@@ -294,7 +285,6 @@ netfs_sreq_ref_traces;
netfs_folio_traces;
netfs_collect_contig_traces;
netfs_donate_traces;
-netfs_folioq_traces;
/*
* Now redefine the EM() and E_() macros to map the enums to the strings that
@@ -398,7 +388,7 @@ TRACE_EVENT(netfs_sreq,
__entry->len = sreq->len;
__entry->transferred = sreq->transferred;
__entry->start = sreq->start;
- __entry->slot = sreq->io_iter.folioq_slot;
+ __entry->slot = sreq->io_iter.bvecq_slot;
),
TP_printk("R=%08x[%x] %s %s f=%03x s=%llx %zx/%zx s=%u e=%d",
@@ -773,29 +763,6 @@ TRACE_EVENT(netfs_collect_stream,
__entry->collected_to, __entry->issued_to)
);
-TRACE_EVENT(netfs_folioq,
- TP_PROTO(const struct folio_queue *fq,
- enum netfs_folioq_trace trace),
-
- TP_ARGS(fq, trace),
-
- TP_STRUCT__entry(
- __field(unsigned int, rreq)
- __field(unsigned int, id)
- __field(enum netfs_folioq_trace, trace)
- ),
-
- TP_fast_assign(
- __entry->rreq = fq ? fq->rreq_id : 0;
- __entry->id = fq ? fq->debug_id : 0;
- __entry->trace = trace;
- ),
-
- TP_printk("R=%08x fq=%x %s",
- __entry->rreq, __entry->id,
- __print_symbolic(__entry->trace, netfs_folioq_traces))
- );
-
TRACE_EVENT(netfs_read_progress_at,
TP_PROTO(const struct netfs_io_request *rreq),
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index d870bc5e50bc8..63fd6156ffa7c 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -6803,8 +6803,6 @@ static const struct bpf_raw_tp_null_args raw_tp_null_args[] = {
/* amdgpu */
{ "amdgpu_vm_bo_map", 0x1 },
{ "amdgpu_vm_bo_unmap", 0x1 },
- /* netfs */
- { "netfs_folioq", 0x1 },
/* xfs from xfs_defer_pending_class */
{ "xfs_defer_create_intent", 0x1 },
{ "xfs_defer_cancel_list", 0x1 },
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 2072c04e99d05..7dcb81ba2a73a 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -538,37 +538,38 @@ static void iov_iter_iovec_advance(struct iov_iter *i, size_t size)
i->__iov = iov;
}
-static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)
+static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)
{
- const struct folio_queue *folioq = i->folioq;
- unsigned int slot = i->folioq_slot;
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
if (!i->count)
return;
- i->count -= size;
-
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- slot = 0;
- }
+ i->count -= by;
- size += i->iov_offset; /* From beginning of current segment. */
+ by += i->iov_offset; /* From beginning of current segment. */
do {
- size_t fsize = folioq_folio_size(folioq, slot);
+ size_t len;
- if (likely(size < fsize))
- break;
- size -= fsize;
- slot++;
- if (slot >= folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
+ if (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
slot = 0;
+ continue;
}
- } while (size);
- i->iov_offset = size;
- i->folioq_slot = slot;
- i->folioq = folioq;
+ len = bq->bv[slot].bv_len;
+
+ if (likely(by < len))
+ break;
+ by -= len;
+ slot++;
+ } while (by);
+
+ i->iov_offset = by;
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
}
void iov_iter_advance(struct iov_iter *i, size_t size)
@@ -583,38 +584,39 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
iov_iter_iovec_advance(i, size);
} else if (iov_iter_is_bvec(i)) {
iov_iter_bvec_advance(i, size);
- } else if (iov_iter_is_folioq(i)) {
- iov_iter_folioq_advance(i, size);
+ } else if (iov_iter_is_bvecq(i)) {
+ iov_iter_bvecq_advance(i, size);
} else if (iov_iter_is_discard(i)) {
i->count -= size;
}
}
EXPORT_SYMBOL(iov_iter_advance);
-static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)
+static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)
{
- const struct folio_queue *folioq = i->folioq;
- unsigned int slot = i->folioq_slot;
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
for (;;) {
- size_t fsize;
+ size_t len;
if (slot == 0) {
- folioq = folioq->prev;
- slot = folioq_nr_slots(folioq);
+ bq = bq->prev;
+ slot = bq->nr_slots;
+ continue;
}
slot--;
- fsize = folioq_folio_size(folioq, slot);
- if (unroll <= fsize) {
- i->iov_offset = fsize - unroll;
+ len = bq->bv[slot].bv_len;
+ if (unroll <= len) {
+ i->iov_offset = len - unroll;
break;
}
- unroll -= fsize;
+ unroll -= len;
}
- i->folioq_slot = slot;
- i->folioq = folioq;
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
}
void iov_iter_revert(struct iov_iter *i, size_t unroll)
@@ -648,9 +650,9 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
}
unroll -= n;
}
- } else if (iov_iter_is_folioq(i)) {
+ } else if (iov_iter_is_bvecq(i)) {
i->iov_offset = 0;
- iov_iter_folioq_revert(i, unroll);
+ iov_iter_bvecq_revert(i, unroll);
} else { /* same logics for iovec and kvec */
const struct iovec *iov = iter_iov(i);
while (1) {
@@ -678,9 +680,28 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return min(i->count, i->bvec->bv_len - i->iov_offset);
}
- if (unlikely(iov_iter_is_folioq(i)))
- return !i->count ? 0 :
- umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
+ if (!i->count)
+ return 0;
+ if (unlikely(iov_iter_is_bvecq(i))) {
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+ size_t offset = i->iov_offset;
+
+ for (;;) {
+ while (slot >= bq->nr_slots) {
+ bq = bq->next;
+ if (!bq)
+ return 0;
+ slot = 0;
+ offset = 0;
+ }
+ if (bq->bv[slot].bv_len > offset)
+ break;
+ slot++;
+ offset = 0;
+ }
+ return min(i->count, bq->bv[slot].bv_len - offset);
+ }
return i->count;
}
EXPORT_SYMBOL(iov_iter_single_seg_count);
@@ -718,34 +739,33 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction,
EXPORT_SYMBOL(iov_iter_bvec);
/**
- * iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue
+ * iov_iter_bvec_queue - Initialise an I/O iterator to use a segmented bvec queue
* @i: The iterator to initialise.
* @direction: The direction of the transfer.
- * @folioq: The starting point in the folio queue.
- * @first_slot: The first slot in the folio queue to use
- * @offset: The offset into the folio in the first slot to start at
+ * @bvecq: The starting point in the bvec queue.
+ * @first_slot: The first slot in the bvec queue to use
+ * @offset: The offset into the bvec in the first slot to start at
* @count: The size of the I/O buffer in bytes.
*
- * Set up an I/O iterator to either draw data out of the pages attached to an
- * inode or to inject data into those pages. The pages *must* be prevented
- * from evaporation, either by taking a ref on them or locking them by the
- * caller.
+ * Set up an I/O iterator to either draw data out of the buffers attached to an
+ * inode or to inject data into those buffers. The pages *must* be prevented
+ * from evaporation, either by the caller.
*/
-void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
- const struct folio_queue *folioq, unsigned int first_slot,
- unsigned int offset, size_t count)
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq, unsigned int first_slot,
+ unsigned int offset, size_t count)
{
- BUG_ON(direction & ~1);
+ WARN_ON(direction & ~(READ | WRITE));
*i = (struct iov_iter) {
- .iter_type = ITER_FOLIOQ,
- .data_source = direction,
- .folioq = folioq,
- .folioq_slot = first_slot,
- .count = count,
- .iov_offset = offset,
+ .iter_type = ITER_BVECQ,
+ .data_source = direction,
+ .bvecq = bvecq,
+ .bvecq_slot = first_slot,
+ .count = count,
+ .iov_offset = offset,
};
}
-EXPORT_SYMBOL(iov_iter_folio_queue);
+EXPORT_SYMBOL(iov_iter_bvec_queue);
/**
* iov_iter_xarray - Initialise an I/O iterator to use the pages in an xarray
@@ -839,6 +859,39 @@ static unsigned long iov_iter_alignment_bvec(const struct iov_iter *i)
return res;
}
+static unsigned long iov_iter_alignment_bvecq(const struct iov_iter *iter)
+{
+ const struct bvecq *bq;
+ unsigned long res = 0;
+ unsigned int slot = iter->bvecq_slot;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ if (!size)
+ return res;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t part = min(bvec->bv_len - skip, size);
+
+ if (part) {
+ res |= bvec->bv_offset + skip;
+ res |= part;
+ }
+
+ size -= part;
+ if (size == 0)
+ return res;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+
+ return res;
+}
+
unsigned long iov_iter_alignment(const struct iov_iter *i)
{
if (likely(iter_is_ubuf(i))) {
@@ -854,10 +907,10 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return iov_iter_alignment_bvec(i);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_alignment_bvecq(i);
- /* With both xarray and folioq types, we're dealing with whole folios. */
- if (iov_iter_is_folioq(i))
- return i->iov_offset | i->count;
+ /* With the xarray type, we're dealing with whole folios. */
if (iov_iter_is_xarray(i))
return (i->xarray_start + i->iov_offset) | i->count;
@@ -910,62 +963,156 @@ static int want_pages_array(struct page ***res, size_t size,
return count;
}
-static ssize_t iter_folioq_get_pages(struct iov_iter *iter,
- struct page ***ppages, size_t maxsize,
- unsigned maxpages, size_t *_start_offset)
+/*
+ * Count the number of virtually contiguous pages coming up next in an
+ * ITER_BVECQ iterator, up to the specified maxima.
+ */
+static unsigned int iter_count_bvecq_pages(const struct iov_iter *iter,
+ size_t maxsize,
+ unsigned int maxpages)
{
- const struct folio_queue *folioq = iter->folioq;
+ const struct bvecq *bvecq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
+ ssize_t remain = umin(maxsize, iter->count);
+ size_t count = 0, offset = iter->iov_offset;
+
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
+
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(remain > 0);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+
+ bv = &bvecq->bv[slot++];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ /* bv_page is not allowed to be NULL unless bv_len == 0. */
+ if (WARN_ON_ONCE(!bv->bv_page && blen > 0))
+ break;
+ if (!PAGE_ALIGNED(boff) && count > 0)
+ break;
+
+ boff += offset;
+ blen -= offset;
+ offset = 0;
+ if (!blen)
+ continue;
+
+ blen = umin(blen, remain);
+ remain -= blen;
+ blen += offset_in_page(boff);
+ count += DIV_ROUND_UP(blen, PAGE_SIZE);
+
+ if (!PAGE_ALIGNED(blen))
+ break;
+ } while (remain > 0 && count < maxpages);
+
+ return min(count, maxpages);
+}
+
+/*
+ * Get a list of virtually contiguous pages from an ITER_BVECQ iterator and pin
+ * them.
+ */
+static ssize_t iter_bvecq_get_pages(struct iov_iter *iter,
+ struct page ***ppages, size_t maxsize,
+ unsigned int maxpages, size_t *_start_offset)
+{
+ const struct bvecq *bvecq = iter->bvecq;
struct page **pages;
- unsigned int slot = iter->folioq_slot;
- size_t extracted = 0, count = iter->count, iov_offset = iter->iov_offset;
+ unsigned int slot = iter->bvecq_slot, nr = 0;
+ size_t extracted = 0, offset = iter->iov_offset;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- slot = 0;
- if (WARN_ON(iov_offset != 0))
- return -EIO;
+ for (;;) {
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next)
+ return 0;
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+ if (offset >= bvecq->bv[slot].bv_len) {
+ slot++;
+ offset = 0;
+ continue;
+ }
+ break;
}
- maxpages = want_pages_array(ppages, maxsize, iov_offset & ~PAGE_MASK, maxpages);
+ /* Count the next run of virtually contiguous pages. */
+ maxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);
+ if (!maxpages)
+ return 0;
+
+ maxpages = want_pages_array(ppages, maxsize,
+ (bvecq->bv[slot].bv_offset + offset) & ~PAGE_MASK,
+ maxpages);
if (!maxpages)
return -ENOMEM;
- *_start_offset = iov_offset & ~PAGE_MASK;
pages = *ppages;
- for (;;) {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t offset = iov_offset, fsize = folioq_folio_size(folioq, slot);
- size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
- if (offset < fsize) {
- part = umin(part, umin(maxsize - extracted, fsize - offset));
- count -= part;
- iov_offset += part;
- extracted += part;
+ /* Now transcribe the page pointers. */
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
- *pages = folio_page(folio, offset / PAGE_SIZE);
- get_page(*pages);
- pages++;
- maxpages--;
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(extracted < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
}
- if (maxpages == 0 || extracted >= maxsize)
- break;
+ bv = &bvecq->bv[slot];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ /* bv_page is not allowed to be NULL unless bv_len == 0. */
+
+ if (offset < blen) {
+ size_t poff = (boff + offset) % PAGE_SIZE;
+ size_t part = min(maxsize - extracted, blen - offset);
+ size_t pix = (boff + offset) / PAGE_SIZE;
+
+ if (poff + part > PAGE_SIZE)
+ part = PAGE_SIZE - poff;
+
+ if (!extracted)
+ *_start_offset = poff;
- if (iov_offset >= fsize) {
- iov_offset = 0;
+ /* Get a ref on the page if it's not a slab page. */
+ if (!PageSlab(bv->bv_page + pix))
+ get_page(bv->bv_page + pix);
+
+ pages[nr++] = bv->bv_page + pix;
+ offset += part;
+ extracted += part;
+ }
+
+ if (offset >= blen) {
+ offset = 0;
slot++;
- if (slot == folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
- slot = 0;
- }
}
- }
+ } while (nr < maxpages && extracted < maxsize);
- iter->count = count;
- iter->iov_offset = iov_offset;
- iter->folioq = folioq;
- iter->folioq_slot = slot;
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= extracted;
return extracted;
}
@@ -1120,8 +1267,8 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
}
return maxsize;
}
- if (iov_iter_is_folioq(i))
- return iter_folioq_get_pages(i, pages, maxsize, maxpages, start);
+ if (iov_iter_is_bvecq(i))
+ return iter_bvecq_get_pages(i, pages, maxsize, maxpages, start);
if (iov_iter_is_xarray(i))
return iter_xarray_get_pages(i, pages, maxsize, maxpages, start);
return -EFAULT;
@@ -1192,6 +1339,38 @@ static int bvec_npages(const struct iov_iter *i, int maxpages)
return npages;
}
+static size_t iov_npages_bvecq(const struct iov_iter *iter, size_t maxpages)
+{
+ const struct bvecq *bq;
+ unsigned int slot = iter->bvecq_slot;
+ size_t npages = 0;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t offs = (bvec->bv_offset + skip) % PAGE_SIZE;
+ size_t part = min(bvec->bv_len - skip, size);
+
+ if (part) {
+ npages += DIV_ROUND_UP(offs + part, PAGE_SIZE);
+ if (npages >= maxpages)
+ goto out;
+ }
+
+ size -= part;
+ if (!size)
+ goto out;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+out:
+ return umin(npages, maxpages);
+}
+
int iov_iter_npages(const struct iov_iter *i, int maxpages)
{
if (unlikely(!i->count))
@@ -1206,11 +1385,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
return iov_npages(i, maxpages);
if (iov_iter_is_bvec(i))
return bvec_npages(i, maxpages);
- if (iov_iter_is_folioq(i)) {
- unsigned offset = i->iov_offset % PAGE_SIZE;
- int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
- return min(npages, maxpages);
- }
+ if (iov_iter_is_bvecq(i))
+ return iov_npages_bvecq(i, maxpages);
if (iov_iter_is_xarray(i)) {
unsigned offset = (i->xarray_start + i->iov_offset) % PAGE_SIZE;
int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
@@ -1494,64 +1670,86 @@ void iov_iter_restore(struct iov_iter *i, struct iov_iter_state *state)
EXPORT_SYMBOL_FOR_MODULES(iov_iter_restore, "vmw_vsock_virtio_transport_common");
/*
- * Extract a list of contiguous pages from an ITER_FOLIOQ iterator. This does
- * not get references on the pages, nor does it get a pin on them.
+ * Extract a list of virtually contiguous pages from an ITER_BVECQ iterator.
+ * This does not get references on the pages, nor does it get a pin on them.
*/
-static ssize_t iov_iter_extract_folioq_pages(struct iov_iter *i,
- struct page ***pages, size_t maxsize,
- unsigned int maxpages,
- iov_iter_extraction_t extraction_flags,
- size_t *offset0)
+static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,
+ struct page ***pages, size_t maxsize,
+ unsigned int maxpages,
+ iov_iter_extraction_t extraction_flags,
+ size_t *offset0)
{
- const struct folio_queue *folioq = i->folioq;
+ const struct bvecq *bvecq;
struct page **p;
- unsigned int nr = 0;
- size_t extracted = 0, offset, slot = i->folioq_slot;
+ unsigned int slot, nr = 0;
+ size_t extracted = 0, offset;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- slot = 0;
- if (WARN_ON(i->iov_offset != 0))
- return -EIO;
- }
+ /* Count the next run of virtually contiguous pages. */
+ maxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);
+ if (!maxpages)
+ return 0;
- offset = i->iov_offset & ~PAGE_MASK;
- *offset0 = offset;
+ if (!*pages) {
+ *pages = kvmalloc_array(maxpages, sizeof(struct page *), GFP_KERNEL);
+ if (!*pages)
+ return -ENOMEM;
+ }
- maxpages = want_pages_array(pages, maxsize, offset, maxpages);
- if (!maxpages)
- return -ENOMEM;
p = *pages;
- for (;;) {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t offset = i->iov_offset, fsize = folioq_folio_size(folioq, slot);
- size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
- if (offset < fsize) {
- part = umin(part, umin(maxsize - extracted, fsize - offset));
- i->count -= part;
- i->iov_offset += part;
- extracted += part;
+ /* Now transcribe the page pointers. */
+ extracted = 0;
+ bvecq = iter->bvecq;
+ offset = iter->iov_offset;
+ slot = iter->bvecq_slot;
+
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
- p[nr++] = folio_page(folio, offset / PAGE_SIZE);
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(extracted < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
}
- if (nr >= maxpages || extracted >= maxsize)
- break;
+ bv = &bvecq->bv[slot];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
- if (i->iov_offset >= fsize) {
- i->iov_offset = 0;
+ /* bv_page is not allowed to be NULL unless bv_len == 0. */
+
+ if (offset < blen) {
+ size_t part = umin(maxsize - extracted, blen - offset);
+ size_t poff = (boff + offset) % PAGE_SIZE;
+ size_t pix = (boff + offset) / PAGE_SIZE;
+
+ if (poff + part > PAGE_SIZE)
+ part = PAGE_SIZE - poff;
+
+ if (!extracted)
+ *offset0 = poff;
+
+ p[nr++] = bv->bv_page + pix;
+ offset += part;
+ extracted += part;
+ }
+
+ if (offset >= blen) {
+ offset = 0;
slot++;
- if (slot == folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
- slot = 0;
- }
}
- }
+ } while (nr < maxpages && extracted < maxsize);
- i->folioq = folioq;
- i->folioq_slot = slot;
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= extracted;
return extracted;
}
@@ -1817,8 +2015,8 @@ static ssize_t iov_iter_extract_user_pages(struct iov_iter *i,
* added to the pages, but refs will not be taken.
* iov_iter_extract_will_pin() will return true.
*
- * (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_FOLIOQ or ITER_XARRAY, the
- * pages are merely listed; no extra refs or pins are obtained.
+ * (*) If the iterator is ITER_KVEC, ITER_BVEC, ITER_XARRAY, the pages are
+ * merely listed; no extra refs or pins are obtained.
* iov_iter_extract_will_pin() will return 0.
*
* Note also:
@@ -1853,10 +2051,10 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,
return iov_iter_extract_bvec_pages(i, pages, maxsize,
maxpages, extraction_flags,
offset0);
- if (iov_iter_is_folioq(i))
- return iov_iter_extract_folioq_pages(i, pages, maxsize,
- maxpages, extraction_flags,
- offset0);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_extract_bvecq_pages(i, pages, maxsize,
+ maxpages, extraction_flags,
+ offset0);
if (iov_iter_is_xarray(i))
return iov_iter_extract_xarray_pages(i, pages, maxsize,
maxpages, extraction_flags,
diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index 6ea40d2e62475..b9a7298306d95 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -10,8 +10,8 @@
#include <linux/highmem.h>
#include <linux/kmemleak.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/uio.h>
-#include <linux/folio_queue.h>
/**
* sg_nents - return total count of entries in scatterlist
@@ -1268,61 +1268,59 @@ static ssize_t extract_kvec_to_sg(struct iov_iter *iter,
}
/*
- * Extract up to sg_max folios from an FOLIOQ-type iterator and add them to
+ * Extract up to sg_max folios from an BVECQ-type iterator and add them to
* the scatterlist. The pages are not pinned.
*/
-static ssize_t extract_folioq_to_sg(struct iov_iter *iter,
+static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,
ssize_t maxsize,
struct sg_table *sgtable,
unsigned int sg_max,
iov_iter_extraction_t extraction_flags)
{
- const struct folio_queue *folioq = iter->folioq;
+ const struct bvecq *bvecq = iter->bvecq;
struct scatterlist *sg = sgtable->sgl + sgtable->nents;
- unsigned int slot = iter->folioq_slot;
+ unsigned int slot = iter->bvecq_slot;
ssize_t ret = 0;
size_t offset = iter->iov_offset;
- BUG_ON(!folioq);
+ maxsize = umin(maxsize, iov_iter_count(iter));
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- if (WARN_ON_ONCE(!folioq))
- return 0;
- slot = 0;
- }
-
- do {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t fsize = folioq_folio_size(folioq, slot);
+ while (sg_max > 0 && ret < maxsize) {
+ const struct bio_vec *bv;
+ size_t blen, part;
- if (offset < fsize) {
- size_t part = umin(maxsize - ret, fsize - offset);
-
- sg_set_page(sg, folio_page(folio, 0), part, offset);
- sgtable->nents++;
- sg++;
- sg_max--;
- offset += part;
- ret += part;
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(ret < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
}
- if (offset >= fsize) {
+ bv = &bvecq->bv[slot];
+ blen = bv->bv_len;
+
+ if (offset >= blen) {
offset = 0;
slot++;
- if (slot >= folioq_nr_slots(folioq)) {
- if (!folioq->next) {
- WARN_ON_ONCE(ret < iter->count);
- break;
- }
- folioq = folioq->next;
- slot = 0;
- }
+ continue;
}
- } while (sg_max > 0 && ret < maxsize);
- iter->folioq = folioq;
- iter->folioq_slot = slot;
+ part = umin(maxsize - ret, blen - offset);
+
+ sg_set_page(sg, bv->bv_page, part, bv->bv_offset + offset);
+ sgtable->nents++;
+ sg++;
+ sg_max--;
+ offset += part;
+ ret += part;
+ }
+
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
iter->iov_offset = offset;
iter->count -= ret;
return ret;
@@ -1391,8 +1389,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,
* addition of @sg_max elements.
*
* The pages referred to by UBUF- and IOVEC-type iterators are extracted and
- * pinned; BVEC-, KVEC-, FOLIOQ- and XARRAY-type are extracted but aren't
- * pinned; DISCARD-type is not supported.
+ * pinned; BVEC-, BVECQ-, KVEC-, XARRAY-type are extracted but aren't pinned;
+ * DISCARD-type is not supported.
*
* No end mark is placed on the scatterlist; that's left to the caller.
*
@@ -1424,9 +1422,9 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,
case ITER_KVEC:
return extract_kvec_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
- case ITER_FOLIOQ:
- return extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,
- extraction_flags);
+ case ITER_BVECQ:
+ return extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,
+ extraction_flags);
case ITER_XARRAY:
return extract_xarray_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
diff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c
index 32e42d8c7ca13..a2a83ba613fda 100644
--- a/lib/tests/kunit_iov_iter.c
+++ b/lib/tests/kunit_iov_iter.c
@@ -12,7 +12,7 @@
#include <linux/mm.h>
#include <linux/uio.h>
#include <linux/bvec.h>
-#include <linux/folio_queue.h>
+#include <linux/bvecq.h>
#include <linux/scatterlist.h>
#include <linux/minmax.h>
#include <linux/mman.h>
@@ -382,58 +382,67 @@ static void __init iov_kunit_copy_from_bvec(struct kunit *test)
KUNIT_SUCCEED(test);
}
-static void iov_kunit_destroy_folioq(void *data)
+static void iov_kunit_destroy_bvecq(void *data)
{
- struct folio_queue *folioq, *next;
+ struct bvecq *bq, *next;
- for (folioq = data; folioq; folioq = next) {
- next = folioq->next;
- kfree(folioq);
+ for (bq = data; bq; bq = next) {
+ next = bq->next;
+ /* The pages are freed by vmap with VM_MAP_PUT_PAGES. */
+ kfree(bq);
}
}
-static void __init iov_kunit_load_folioq(struct kunit *test,
+static struct bvecq *iov_kunit_alloc_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = kzalloc(struct_size(bq, __bv, max_slots), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, bq);
+ bq->max_slots = max_slots;
+ bq->bv = bq->__bv;
+ bq->inline_bv = true;
+ return bq;
+}
+
+static struct bvecq *iov_kunit_create_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = iov_kunit_alloc_bvecq(test, max_slots);
+ kunit_add_action_or_reset(test, iov_kunit_destroy_bvecq, bq);
+ return bq;
+}
+
+static void __init iov_kunit_load_bvecq(struct kunit *test,
struct iov_iter *iter, int dir,
- struct folio_queue *folioq,
+ struct bvecq *bq_head,
struct page **pages, size_t npages)
{
- struct folio_queue *p = folioq;
+ struct bvecq *bq = bq_head;
size_t size = 0;
- int i;
- for (i = 0; i < npages; i++) {
- if (folioq_full(p)) {
- p->next = kzalloc_obj(struct folio_queue);
- KUNIT_ASSERT_NOT_ERR_OR_NULL(test, p->next);
- folioq_init(p->next, 0);
- p->next->prev = p;
- p = p->next;
+ for (int i = 0; i < npages; i++) {
+ if (bq->nr_slots >= bq->max_slots) {
+ bq->next = iov_kunit_alloc_bvecq(test, 13);
+ bq->next->prev = bq;
+ bq = bq->next;
}
- folioq_append(p, page_folio(pages[i]));
+ bvec_set_page(&bq->bv[bq->nr_slots], pages[i], PAGE_SIZE, 0);
+ bq->nr_slots++;
size += PAGE_SIZE;
}
- iov_iter_folio_queue(iter, dir, folioq, 0, 0, size);
-}
-
-static struct folio_queue *iov_kunit_create_folioq(struct kunit *test)
-{
- struct folio_queue *folioq;
-
- folioq = kzalloc_obj(struct folio_queue);
- KUNIT_ASSERT_NOT_ERR_OR_NULL(test, folioq);
- kunit_add_action_or_reset(test, iov_kunit_destroy_folioq, folioq);
- folioq_init(folioq, 0);
- return folioq;
+ iov_iter_bvec_queue(iter, dir, bq_head, 0, 0, size);
}
/*
- * Test copying to a ITER_FOLIOQ-type iterator.
+ * Test copying to a ITER_BVECQ-type iterator.
*/
-static void __init iov_kunit_copy_to_folioq(struct kunit *test)
+static void __init iov_kunit_copy_to_bvecq(struct kunit *test)
{
const struct kvec_test_range *pr;
struct iov_iter iter;
- struct folio_queue *folioq;
+ struct bvecq *bq;
struct page **spages, **bpages;
u8 *scratch, *buffer;
size_t bufsize, npages, size, copied;
@@ -442,7 +451,7 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)
bufsize = 0x100000;
npages = bufsize / PAGE_SIZE;
- folioq = iov_kunit_create_folioq(test);
+ bq = iov_kunit_create_bvecq(test, 13);
scratch = iov_kunit_create_buffer(test, &spages, npages);
for (i = 0; i < bufsize; i++)
@@ -451,20 +460,19 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)
buffer = iov_kunit_create_buffer(test, &bpages, npages);
memset(buffer, 0, bufsize);
- iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
i = 0;
for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
size = pr->to - pr->from;
KUNIT_ASSERT_LE(test, pr->to, bufsize);
- iov_iter_folio_queue(&iter, READ, folioq, 0, 0, pr->to);
+ iov_iter_bvec_queue(&iter, READ, bq, 0, 0, pr->to);
iov_iter_advance(&iter, pr->from);
copied = copy_to_iter(scratch + i, size, &iter);
KUNIT_EXPECT_EQ(test, copied, size);
KUNIT_EXPECT_EQ(test, iter.count, 0);
- KUNIT_EXPECT_EQ(test, iter.iov_offset, pr->to % PAGE_SIZE);
i += size;
if (test->status == KUNIT_FAILURE)
goto stop;
@@ -489,13 +497,13 @@ static void __init iov_kunit_copy_to_folioq(struct kunit *test)
}
/*
- * Test copying from a ITER_FOLIOQ-type iterator.
+ * Test copying from a ITER_BVECQ-type iterator.
*/
-static void __init iov_kunit_copy_from_folioq(struct kunit *test)
+static void __init iov_kunit_copy_from_bvecq(struct kunit *test)
{
const struct kvec_test_range *pr;
struct iov_iter iter;
- struct folio_queue *folioq;
+ struct bvecq *bq;
struct page **spages, **bpages;
u8 *scratch, *buffer;
size_t bufsize, npages, size, copied;
@@ -504,7 +512,7 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)
bufsize = 0x100000;
npages = bufsize / PAGE_SIZE;
- folioq = iov_kunit_create_folioq(test);
+ bq = iov_kunit_create_bvecq(test, 13);
buffer = iov_kunit_create_buffer(test, &bpages, npages);
for (i = 0; i < bufsize; i++)
@@ -513,20 +521,19 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)
scratch = iov_kunit_create_buffer(test, &spages, npages);
memset(scratch, 0, bufsize);
- iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
i = 0;
for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
size = pr->to - pr->from;
KUNIT_ASSERT_LE(test, pr->to, bufsize);
- iov_iter_folio_queue(&iter, WRITE, folioq, 0, 0, pr->to);
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
iov_iter_advance(&iter, pr->from);
copied = copy_from_iter(scratch + i, size, &iter);
KUNIT_EXPECT_EQ(test, copied, size);
KUNIT_EXPECT_EQ(test, iter.count, 0);
- KUNIT_EXPECT_EQ(test, iter.iov_offset, pr->to % PAGE_SIZE);
i += size;
}
@@ -868,13 +875,13 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)
}
/*
- * Test the extraction of ITER_FOLIOQ-type iterators.
+ * Test the extraction of ITER_BVECQ-type iterators.
*/
-static void __init iov_kunit_extract_pages_folioq(struct kunit *test)
+static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)
{
const struct kvec_test_range *pr;
- struct folio_queue *folioq;
struct iov_iter iter;
+ struct bvecq *bq;
struct page **bpages, *pagelist[8], **pages = pagelist;
ssize_t len;
size_t bufsize, size = 0, npages;
@@ -883,17 +890,17 @@ static void __init iov_kunit_extract_pages_folioq(struct kunit *test)
bufsize = 0x100000;
npages = bufsize / PAGE_SIZE;
- folioq = iov_kunit_create_folioq(test);
+ bq = iov_kunit_create_bvecq(test, 13);
iov_kunit_create_buffer(test, &bpages, npages);
- iov_kunit_load_folioq(test, &iter, READ, folioq, bpages, npages);
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
from = pr->from;
size = pr->to - from;
KUNIT_ASSERT_LE(test, pr->to, bufsize);
- iov_iter_folio_queue(&iter, WRITE, folioq, 0, 0, pr->to);
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
iov_iter_advance(&iter, from);
do {
@@ -1173,23 +1180,6 @@ static void __init iov_kunit_iter_to_sg_bvec(struct kunit *test)
iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
}
-static void __init iov_kunit_iter_to_sg_folioq(struct kunit *test)
-{
- struct iov_kunit_iter_to_sg_data data;
- struct folio_queue *folioq;
- struct iov_iter iter;
- size_t bufsize;
-
- bufsize = 0x200000;
- iov_kunit_iter_to_sg_init(test, bufsize, false, &data);
-
- folioq = iov_kunit_create_folioq(test);
- iov_kunit_load_folioq(test, &iter, READ, folioq, data.pages,
- data.npages);
-
- iov_kunit_iter_to_sg_check(test, &iter, bufsize, &data);
-}
-
static void __init iov_kunit_iter_to_sg_xarray(struct kunit *test)
{
struct iov_kunit_iter_to_sg_data data;
@@ -1226,17 +1216,16 @@ static struct kunit_case __refdata iov_kunit_cases[] = {
KUNIT_CASE(iov_kunit_copy_from_kvec),
KUNIT_CASE(iov_kunit_copy_to_bvec),
KUNIT_CASE(iov_kunit_copy_from_bvec),
- KUNIT_CASE(iov_kunit_copy_to_folioq),
- KUNIT_CASE(iov_kunit_copy_from_folioq),
+ KUNIT_CASE(iov_kunit_copy_to_bvecq),
+ KUNIT_CASE(iov_kunit_copy_from_bvecq),
KUNIT_CASE(iov_kunit_copy_to_xarray),
KUNIT_CASE(iov_kunit_copy_from_xarray),
KUNIT_CASE(iov_kunit_extract_pages_kvec),
KUNIT_CASE(iov_kunit_extract_pages_bvec),
- KUNIT_CASE(iov_kunit_extract_pages_folioq),
+ KUNIT_CASE(iov_kunit_extract_pages_bvecq),
KUNIT_CASE(iov_kunit_extract_pages_xarray),
KUNIT_CASE(iov_kunit_iter_to_sg_kvec),
KUNIT_CASE(iov_kunit_iter_to_sg_bvec),
- KUNIT_CASE(iov_kunit_iter_to_sg_folioq),
KUNIT_CASE(iov_kunit_iter_to_sg_xarray),
KUNIT_CASE(iov_kunit_iter_to_sg_ubuf),
{}
|
| 23/3 | 2026/09/30 13:48 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 60719 output: 21 thoughts: 5154 Thoughts: **Review of `folio_queue` to `bvecq` Patch and Potential Uninitialized Memory Risks**
Okay, let's break down this patch. The core change is replacing `folio_queue` with `bvecq` (segmented queue of `bio_vec`) across various subsystems. My initial concern, as always, is uninitialized memory. Let's dig in.
The patch seems to be a structural change, and the key is in `fs/netfs/bvecq.c`. `bvecq_alloc_one` appears to allocate a `bvecq` struct along with an array `__bv[]`, but the array itself is *not* initialized. The important line is `*bq = (struct bvecq){...}` which zero-initializes the struct, but this initialization *does not* extend to the flexible array member `__bv[]`.
Now, where could this uninitialized memory bite us? `bvecq_alloc_buffer2` is where things get interesting. This function allocates the pages to populate `__bv[]`. The `bvecq` struct *is* zeroed by construction, but the `__bv[]` array that stores the `bio_vec`s is initially uninitialized. However, within `bvecq_alloc_buffer2`, it is cleared with `memset`. The pages array pointer is overlaid to the end of the `__bv[]`. The pointer arithmetic is clever, reusing unused space and then setting the pages via `bvec_set_page`. This reuse is crucial to avoid having to allocate a separate temporary array for `alloc_pages_bulk`.
Now, about `rolling_buffer_bulk_load_from_ra`. Here, `bvecq_alloc_one` is called. It is not `kzalloc`, which means the allocated `bv[]` array is not zero-initialized, only the struct itself. Subsequent calls to `bvec_set_folio` populate the relevant slots. Later, the loop in this function correctly bounds its iterations based on the filled slots. The code checks boundaries against `nr_slots` which mitigates risks.
I need to confirm all the locations where `bvecq` is utilized, but it seems there is enough boundary checks. My overall assessment is that the patch *carefully* manages the memory, and most access patterns seem safe, *provided* the `nr_slots` and `max_slots` members are correctly managed, which is consistently done. Also, it uses clever pointer arithmetic for memory reuse, avoiding unnecessary allocation and avoiding uninitialized reads.
Also, I am seeing some code such as `afs_init_new_symlink`, which specifically zero-fills the remaining space on a page after a smaller write. This indicates an awareness of potential issues and proactive mitigation.
Further review will be necessary to examine all code paths and make a definitive statement, but my initial impression is that the patch is reasonably safe regarding uninitialized memory. I'll need to double check the calling conventions of all of these functions to ensure that this isn't a problem, but it looks good from a first scan.
|
| 24/3 | 2026/09/30 13:49 | tool | grepper |
0mArgs:{
"Expression": "unsigned long alloc_pages_bulk"
}Results: {
"Output": "include/linux/gfp.h=207=struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int preferred_nid,\n--\ninclude/linux/gfp.h-210-\ninclude/linux/gfp.h:211:unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,\ninclude/linux/gfp.h-212-\t\t\t\tnodemask_t *nodemask, int nr_pages,\n--\ninclude/linux/gfp.h=216=void free_pages_bulk(struct page **page_array, unsigned long nr_pages);\ninclude/linux/gfp.h-217-\ninclude/linux/gfp.h:218:unsigned long alloc_pages_bulk_mempolicy_noprof(gfp_t gfp,\ninclude/linux/gfp.h-219-\t\t\t\tunsigned long nr_pages,\n--\nmm/mempolicy.c=2589=EXPORT_SYMBOL(folio_alloc_noprof);\nmm/mempolicy.c-2590-\nmm/mempolicy.c:2591:static unsigned long alloc_pages_bulk_interleave(gfp_t gfp,\nmm/mempolicy.c-2592-\t\tstruct mempolicy *pol, unsigned long nr_pages,\n--\nmm/mempolicy.c-2625-\nmm/mempolicy.c:2626:static unsigned long alloc_pages_bulk_weighted_interleave(gfp_t gfp,\nmm/mempolicy.c-2627-\t\tstruct mempolicy *pol, unsigned long nr_pages,\n--\nmm/mempolicy.c-2743-\nmm/mempolicy.c:2744:static unsigned long alloc_pages_bulk_preferred_many(gfp_t gfp, int nid,\nmm/mempolicy.c-2745-\t\tstruct mempolicy *pol, unsigned long nr_pages,\n--\nmm/mempolicy.c-2769- */\nmm/mempolicy.c:2770:unsigned long alloc_pages_bulk_mempolicy_noprof(gfp_t gfp,\nmm/mempolicy.c-2771-\t\tunsigned long nr_pages, struct page **page_array)\n--\nmm/page_alloc.c=5091=static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order,\n--\nmm/page_alloc.c-5159- */\nmm/page_alloc.c:5160:unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,\nmm/page_alloc.c-5161-\t\t\tnodemask_t *nodemask, int nr_pages,\n"
} |
| 25/3 | 2026/09/30 13:49 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 67253 output: 36 thoughts: 20 Thoughts: Here's my attempt at summarizing the thought process, tailored for an expert audience: **Diving into `alloc_pages_bulk_noprof` Implementation** Alright, let's take a look at `alloc_pages_bulk_noprof`. I need to get a better grasp of this function, its role, and how it differs (or doesn't) from the more general `alloc_pages`. My initial thought is to go straight to the implementation itself, or perhaps, depending on the complexity, I'll start with the comments, if there are any that are helpful, to get a high-level picture of what's going on. I'm especially interested in understanding the trade-offs it's making – what's being sacrificed to achieve "noprof" behavior? Is it avoiding some bookkeeping overhead? Are there implications for memory allocation policies or NUMA affinity? What specific optimizations does it entail, and how do they impact performance in various scenarios? Understanding this function thoroughly will help me optimize and troubleshoot any memory-related issues in my code that uses it. |
| 26/3 | 2026/09/30 13:49 | tool | read-file |
0mArgs:{
"File": "mm/page_alloc.c",
"FirstLine": 5145,
"LineCount": 40
}Results: {
"Contents": "5145:\t * @page_array: Array to store the pages\n5146:\t *\n5147:\t * This is a batched version of the page allocator that attempts to allocate\n5148:\t * @nr_pages quickly. Pages are added to @page_array.\n5149:\t *\n5150:\t * Note that only the elements in @page_array that were cleared to %NULL on\n5151:\t * entry are populated with newly allocated pages. @nr_pages is the maximum\n5152:\t * number of pages that will be stored in the array.\n5153:\t *\n5154:\t * Returns the number of pages in @page_array, including ones already\n5155:\t * allocated on entry. This can be less than the number requested in @nr_pages,\n5156:\t * but all empty slots are filled from the beginning. I.e., if all slots in\n5157:\t * @page_array were set to %NULL on entry, the slots from 0 to the return value\n5158:\t * - 1 will be filled.\n5159:\t */\n5160:\tunsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,\n5161:\t\t\t\tnodemask_t *nodemask, int nr_pages,\n5162:\t\t\t\tstruct page **page_array)\n5163:\t{\n5164:\t\tstruct page *page;\n5165:\t\tstruct zone *zone;\n5166:\t\tstruct zoneref *z;\n5167:\t\tstruct per_cpu_pages *pcp;\n5168:\t\tstruct list_head *pcp_list;\n5169:\t\tstruct alloc_context ac;\n5170:\t\tunsigned int alloc_flags = ALLOC_WMARK_LOW;\n5171:\t\tint nr_populated = 0, nr_account = 0;\n5172:\t\n5173:\t\t/*\n5174:\t\t * Skip populated array elements to determine if any pages need\n5175:\t\t * to be allocated before disabling IRQs.\n5176:\t\t */\n5177:\t\twhile (nr_populated \u003c nr_pages \u0026\u0026 page_array[nr_populated])\n5178:\t\t\tnr_populated++;\n5179:\t\n5180:\t\t/* No pages requested? */\n5181:\t\tif (unlikely(nr_pages \u003c= 0))\n5182:\t\t\tgoto out;\n5183:\t\n5184:\t\t/* Already populated array? */\n"
} |
| 27/3 | 2026/09/30 13:49 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 68197 output: 19 thoughts: 1558 Thoughts: **Analyzing Potential Page Allocation Issues** Okay, so I'm diving into this code, specifically looking at how pages are allocated and used within this buffer. My initial focus is on the `alloc_pages_bulk` function and its interactions. First, I'm noting lines 5150-5152 which indicate how `alloc_pages_bulk` interacts with `@page_array`. Essentially, pages are allocated from the beginning of the array, but it seems there's an expectation that the array might already have pre-allocated pages. The returned value indicates the number of populated pages, even if some were populated on entry. Lines 5177-5178 then show that the code skips already populated array elements before potentially disabling interrupts. So, I see this is checking which of the pages are populated from the start. Now, looking at `bvecq_alloc_buffer2`, I see a section where `memset` is used to zero out `p->bv` when `p->nr_slots` is zero. This clears the `p->bv` array, making the pages in that `bv` area all null. This is where I start getting interested. I'm wondering, what happens if `alloc_pages_bulk` returns fewer pages than requested (`got < want`)? The code continues to the next iteration. It appears that `pages` is recomputed based on `p->max_slots` and the number of pages already allocated. It also seems that, on the next iteration, the check `p->nr_slots != 0` is true and the `memset` is skipped. This is good because it means that there might be some pages populated by `alloc_pages_bulk` that need to be untouched. I'm starting to get worried though, what about that `bvec_set_page`? This seems to overwrite some stuff! But wait, `bvec_set_page` is writing `struct bio_vec` to the `p->bv` array. So, the question that I was asking previously, is the memory where `pages` is on the next iteration cleared? Well, it's not and that is good! Okay, so looking at the details here. The critical bit is the relationship between the `pages` pointer in one iteration and the next. Specifically, `pages` points to memory in `p->bv`. Because the memset clears `p->bv` if `p->nr_slots == 0`, it means that if the allocation fails, then the subsequent allocation uses `memset`'ed memory. Which is good. So, now I'm thinking about the function `rolling_buffer_bulk_load_from_ra`. Here I see that it calls `__readahead_batch`. And this function allocates `bvecq` and puts it on `roll->head`. And finally it calls `bvec_set_folio`. The `pages` here is calculated using `bq->bv` from the `bvecq`. So, what about `__readahead_batch`? Does it read `pages` or write `pages`? Need to check that definition. |
| 28/3 | 2026/09/30 13:50 | tool | grepper |
0mArgs:{
"Expression": "__readahead_batch"
}Results: {
"Output": "fs/netfs/rolling_buffer.c=75=ssize_t rolling_buffer_bulk_load_from_ra(struct rolling_buffer *roll,\n--\nfs/netfs/rolling_buffer.c-102-\t\tpages -= bq-\u003emax_slots;\nfs/netfs/rolling_buffer.c:103:\t\tnr = __readahead_batch(ractl, pages, bq-\u003emax_slots);\nfs/netfs/rolling_buffer.c-104-\t\tif (WARN_ON_ONCE(!nr))\n--\nfs/squashfs/file.c=574=static void squashfs_readahead(struct readahead_control *ractl)\n--\nfs/squashfs/file.c-606-\nfs/squashfs/file.c:607:\t\tnr_pages = __readahead_batch(ractl, pages, max_pages);\nfs/squashfs/file.c-608-\t\tif (!nr_pages)\n--\ninclude/linux/pagemap.h=1380=static inline bool filemap_range_needs_writeback(struct address_space *mapping,\n--\ninclude/linux/pagemap.h-1396- * implement the -\u003ereadahead method should call readahead_folio() or\ninclude/linux/pagemap.h:1397: * __readahead_batch() in a loop and attempt to start reads into each\ninclude/linux/pagemap.h-1398- * folio in the request.\n--\ninclude/linux/pagemap.h=1511=static inline struct folio *readahead_folio(struct readahead_control *ractl)\n--\ninclude/linux/pagemap.h-1519-\ninclude/linux/pagemap.h:1520:static inline unsigned int __readahead_batch(struct readahead_control *rac,\ninclude/linux/pagemap.h-1521-\t\tstruct page **array, unsigned int array_sz)\n"
} |
| 29/3 | 2026/09/30 13:50 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 70941 output: 38 thoughts: 23 Thoughts: Okay, let's dive into this. Here's my take on those thoughts, as if they're my own: **Analyzing the `__readahead_batch` Function** Right, so the first thing I want to do is get a clear picture of how things work at a low level. I need to understand `__readahead_batch`. Let's navigate to the `include/linux/pagemap.h` file. I know this is a core header, so it will contain the essential structures and function declarations related to the page cache and memory management. I suspect `__readahead_batch` plays a crucial role in improving read performance, likely by prefetching pages. I need to see exactly how it works with the block device layer and how it interacts with the page cache. Is it handling the request queueing? Does it pre-populate the page cache, or something else? I need to look at its arguments, any internal data structures it uses, and, of course, the context in which it's called. This is the starting point for understanding a more efficient read access design, and the overall I/O performance of the system. |
| 30/3 | 2026/09/30 13:50 | tool | read-file |
0mArgs:{
"File": "include/linux/pagemap.h",
"FirstLine": 1515,
"LineCount": 25
}Results: {
"Contents": "1515:\t\tif (folio)\n1516:\t\t\tfolio_put(folio);\n1517:\t\treturn folio;\n1518:\t}\n1519:\t\n1520:\tstatic inline unsigned int __readahead_batch(struct readahead_control *rac,\n1521:\t\t\tstruct page **array, unsigned int array_sz)\n1522:\t{\n1523:\t\tunsigned int i = 0;\n1524:\t\tXA_STATE(xas, \u0026rac-\u003emapping-\u003ei_pages, 0);\n1525:\t\tstruct folio *folio;\n1526:\t\n1527:\t\tBUG_ON(rac-\u003e_batch_count \u003e rac-\u003e_nr_pages);\n1528:\t\trac-\u003e_nr_pages -= rac-\u003e_batch_count;\n1529:\t\trac-\u003e_index += rac-\u003e_batch_count;\n1530:\t\trac-\u003e_batch_count = 0;\n1531:\t\n1532:\t\txas_set(\u0026xas, rac-\u003e_index);\n1533:\t\trcu_read_lock();\n1534:\t\txas_for_each(\u0026xas, folio, rac-\u003e_index + rac-\u003e_nr_pages - 1) {\n1535:\t\t\tif (xas_retry(\u0026xas, folio))\n1536:\t\t\t\tcontinue;\n1537:\t\t\tVM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);\n1538:\t\t\tarray[i++] = folio_page(folio, 0);\n1539:\t\t\trac-\u003e_batch_count += folio_nr_pages(folio);\n"
} |
| 31/3 | 2026/09/30 13:50 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 71655 output: 20 thoughts: 4931 Thoughts: **Analyzing Potential Uninitialized Memory Issues in `bvecq` Data Structure** Okay, so I've been diving deep into this `bvecq` code, specifically the `rolling_buffer_bulk_load_from_ra` function and its related functions, trying to identify any potential for uninitialized memory reads. This is critical for preventing security vulnerabilities and data corruption. Initially, I was focused on how `__readahead_batch` interacts with the `bq->bv` array. My thought process went like this: `__readahead_batch` populates `pages`, and then these are linked to `bq->bv` elements. The pointer arithmetic involving `bq->bv + bq->max_slots` and the subsequent casting to `(struct page **)` looked a little tricky, but the key insight was recognizing that the intervening `-= bq->max_slots` operation cleverly creates a contiguous "scratch space" within the `bq->bv` array, exactly sized for the `pages` array, so long as we take the `bio_vec` size into account. This seems like a reasonable way to allocate space for the pages, without having to allocate space in the heap. But I needed to be sure that the memory locations weren't being overwritten. I was able to verify that the pointer arithmetic meant that the `pages` array was read before `bq->bv`'s memory locations are written to, so there's no overlapping access. It's safe! However, I started thinking about error handling. What if `__readahead_batch` returned a number of pages less than `bq->max_slots`? The loop condition would still be satisfied and we'd correctly set the `bq->nr_slots` and handle this correctly in the `nomem_unlock` block if the page allocation failed. I verified that, when there's an error, the `bq` is added to the list and that all the slots were handled. I then started to think about the allocation path. In particular, the function `bvecq_alloc_buffer2` and the `alloc_pages_bulk` function. I had a hypothesis that there could be cases where `alloc_pages_bulk` allocates fewer pages than requested, potentially leading to uninitialized memory in the `pages` array. If `alloc_pages_bulk` does not allocate all the pages we want, we check if the first slots in `bq->bv` are already initialized (that is, if `nr_slots` is 0). In this case we zero the entire `bv` array, making sure that there is no old data. We then write the slots to the `bq->bv`. Even when `alloc_pages_bulk` does not fill the array, the existing page is cleaned. Also, I realized that if the initial allocation failed, the memory is being properly freed in `bvecq_put`, where uninitialized `pages` is explicitly checked. Looking at `bvecq_expand_buffer` to verify that there are no uninitialized reads here, I see the call to `bvec_set_page` to write to the slot, after which `nr_slots` is incremented. So, no problem here either. Finally, I reviewed `bvec_set_page` and `bvec_set_folio`. In all cases, all pages written into `bvec` are initialized prior to writes, so there are no uninitialized memory reads and no security vulnerabilities. I'm confident that the code is safe from this type of issue. It looks like it is well written and has been carefully analyzed. |
| 32/3 | 2026/09/30 13:50 | tool | grepper |
0mArgs:{
"Expression": "bvec_set_page"
}Results: {
"Output": "block/bio-integrity.c=68=void bio_integrity_alloc_buf(struct bio *bio, gfp_t gfp, bool zero_buffer)\n--\nblock/bio-integrity.c-81-\t\t\tmemset(page_address(page), 0, len);\nblock/bio-integrity.c:82:\t\tbvec_set_page(\u0026bip-\u003ebip_vec[0], page, len, 0);\nblock/bio-integrity.c-83-\t\tbip-\u003ebip_flags |= BIP_MEMPOOL;\nblock/bio-integrity.c-84-\t} else {\nblock/bio-integrity.c:85:\t\tbvec_set_page(\u0026bip-\u003ebip_vec[0], virt_to_page(buf), len,\nblock/bio-integrity.c-86-\t\t\t\toffset_in_page(buf));\n--\nblock/bio-integrity.c=224=int bio_integrity_add_page(struct bio *bio, struct page *page,\n--\nblock/bio-integrity.c-252-\nblock/bio-integrity.c:253:\tbvec_set_page(\u0026bip-\u003ebip_vec[bip-\u003ebip_vcnt], page, len, offset);\nblock/bio-integrity.c-254-\tbip-\u003ebip_vcnt++;\n--\nblock/bio-integrity.c=333=static unsigned int bvec_from_pages(struct bio_vec *bvec, struct page **pages,\n--\nblock/bio-integrity.c-358-\nblock/bio-integrity.c:359:\t\tbvec_set_page(\u0026bvec[nr_bvecs], pages[i], size, offset);\nblock/bio-integrity.c-360-\t\toffset = 0;\n--\nblock/bio.c=1006=void __bio_add_page(struct bio *bio, struct page *page,\n--\nblock/bio.c-1014-\nblock/bio.c:1015:\tbvec_set_page(\u0026bio-\u003ebi_io_vec[bio-\u003ebi_vcnt], page, len, off);\nblock/bio.c-1016-\tbio-\u003ebi_iter.bi_size += len;\n--\ndrivers/block/drbd/drbd_main.c=1518=static int _drbd_send_page(struct drbd_peer_device *peer_device, struct page *page,\n--\ndrivers/block/drbd/drbd_main.c-1539-\ndrivers/block/drbd/drbd_main.c:1540:\t\tbvec_set_page(\u0026bvec, page, len, offset);\ndrivers/block/drbd/drbd_main.c-1541-\t\tiov_iter_bvec(\u0026msg.msg_iter, ITER_SOURCE, \u0026bvec, 1, len);\n--\ndrivers/block/rbd.c=3073=static int setup_copyup_bvecs(struct rbd_obj_request *obj_req, u64 obj_overlap)\n--\ndrivers/block/rbd.c-3091-\ndrivers/block/rbd.c:3092:\t\tbvec_set_page(\u0026obj_req-\u003ecopyup_bvecs[i], page, len, 0);\ndrivers/block/rbd.c-3093-\t\tobj_overlap -= len;\n--\ndrivers/infiniband/sw/siw/siw_qp_tx.c=323=static int siw_tcp_sendpages(struct socket *s, struct page **page, int offset,\n--\ndrivers/infiniband/sw/siw/siw_qp_tx.c-341-\t\t\tmsg.msg_flags \u0026= ~MSG_SPLICE_PAGES;\ndrivers/infiniband/sw/siw/siw_qp_tx.c:342:\t\tbvec_set_page(\u0026bvec, page[i], bytes, offset);\ndrivers/infiniband/sw/siw/siw_qp_tx.c-343-\t\tiov_iter_bvec(\u0026msg.msg_iter, ITER_SOURCE, \u0026bvec, 1, bytes);\n--\ndrivers/nvme/host/tcp.c=1191=static int nvme_tcp_try_send_data(struct nvme_tcp_request *req)\n--\ndrivers/nvme/host/tcp.c-1216-\ndrivers/nvme/host/tcp.c:1217:\t\tbvec_set_page(\u0026bvec, page, len, offset);\ndrivers/nvme/host/tcp.c-1218-\t\tiov_iter_bvec(\u0026msg.msg_iter, ITER_SOURCE, \u0026bvec, 1, len);\n--\ndrivers/nvme/target/io-cmd-file.c=120=static bool nvmet_file_execute_io(struct nvmet_req *req, int ki_flags)\n--\ndrivers/nvme/target/io-cmd-file.c-141-\tfor_each_sg(req-\u003esg, sg, req-\u003esg_cnt, i) {\ndrivers/nvme/target/io-cmd-file.c:142:\t\tbvec_set_page(\u0026req-\u003ef.bvec[bv_cnt], sg_page(sg), sg-\u003elength,\ndrivers/nvme/target/io-cmd-file.c-143-\t\t\t sg-\u003eoffset);\n--\ndrivers/nvme/target/tcp.c=353=static int nvmet_tcp_build_pdu_iovec(struct nvmet_tcp_cmd *cmd)\n--\ndrivers/nvme/target/tcp.c-380-\ndrivers/nvme/target/tcp.c:381:\t\tbvec_set_page(iov, sg_page(sg), iov_len,\ndrivers/nvme/target/tcp.c-382-\t\t\t\tsg-\u003eoffset + sg_offset);\n--\ndrivers/nvme/target/tcp.c=671=static int nvmet_try_send_data(struct nvmet_tcp_cmd *cmd, bool last_in_batch)\n--\ndrivers/nvme/target/tcp.c-688-\ndrivers/nvme/target/tcp.c:689:\t\tbvec_set_page(\u0026bvec, page, left, cmd-\u003eoffset);\ndrivers/nvme/target/tcp.c-690-\t\tiov_iter_bvec(\u0026msg.msg_iter, ITER_SOURCE, \u0026bvec, 1, left);\n--\ndrivers/scsi/iscsi_tcp.c=293=static int iscsi_sw_tcp_xmit_segment(struct iscsi_tcp_conn *tcp_conn,\n--\ndrivers/scsi/iscsi_tcp.c-321-\t\t\toffset += segment-\u003esg_offset + sg-\u003eoffset;\ndrivers/scsi/iscsi_tcp.c:322:\t\t\tbvec_set_page(\u0026bv, sg_page(sg), copy, offset);\ndrivers/scsi/iscsi_tcp.c-323-\t\t} else {\n--\ndrivers/scsi/sd.c=976=static void *sd_set_special_bvec(struct scsi_cmnd *cmd, unsigned int data_len)\n--\ndrivers/scsi/sd.c-993-\t\tclear_highpage(page + n);\ndrivers/scsi/sd.c:994:\tbvec_set_page(\u0026rq-\u003especial_vec, page, data_len, 0);\ndrivers/scsi/sd.c-995-\trq-\u003erq_flags |= RQF_SPECIAL_PAYLOAD;\n--\ndrivers/target/iscsi/iscsi_target_util.c=1090=int iscsit_fe_sendpage_sg(\n--\ndrivers/target/iscsi/iscsi_target_util.c-1139-send_pg:\ndrivers/target/iscsi/iscsi_target_util.c:1140:\t\tbvec_set_page(\u0026bvec, sg_page(sg), sub_len, sg-\u003eoffset + offset);\ndrivers/target/iscsi/iscsi_target_util.c-1141-\t\tiov_iter_bvec(\u0026msghdr.msg_iter, ITER_SOURCE, \u0026bvec, 1, sub_len);\n--\ndrivers/target/target_core_file.c=266=fd_execute_rw_aio(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,\n--\ndrivers/target/target_core_file.c-283-\tfor_each_sg(sgl, sg, sgl_nents, i) {\ndrivers/target/target_core_file.c:284:\t\tbvec_set_page(\u0026aio_cmd-\u003ebvecs[i], sg_page(sg), sg-\u003elength,\ndrivers/target/target_core_file.c-285-\t\t\t sg-\u003eoffset);\n--\ndrivers/target/target_core_file.c=312=static int fd_do_rw(struct se_cmd *cmd, struct file *fd,\n--\ndrivers/target/target_core_file.c-329-\tfor_each_sg(sgl, sg, sgl_nents, i) {\ndrivers/target/target_core_file.c:330:\t\tbvec_set_page(\u0026bvec[i], sg_page(sg), sg-\u003elength, sg-\u003eoffset);\ndrivers/target/target_core_file.c-331-\t\tlen += sg-\u003elength;\n--\ndrivers/target/target_core_file.c=428=fd_execute_write_same(struct se_cmd *cmd)\n--\ndrivers/target/target_core_file.c-462-\tfor (i = 0; i \u003c nolb; i++) {\ndrivers/target/target_core_file.c:463:\t\tbvec_set_page(\u0026bvec[i], sg_page(\u0026cmd-\u003et_data_sg[0]),\ndrivers/target/target_core_file.c-464-\t\t\t cmd-\u003et_data_sg[0].length,\n--\ndrivers/vhost/vringh.c=1039=static int iotlb_translate(const struct vringh *vrh,\n--\ndrivers/vhost/vringh.c-1081-\ndrivers/vhost/vringh.c:1082:\t\t\tbvec_set_page(\u0026bvec[ret], pfn_to_page(pfn), io_len,\ndrivers/vhost/vringh.c-1083-\t\t\t\t io_addr \u0026 (PAGE_SIZE - 1));\n--\nfs/ceph/file.c=104=static ssize_t __iter_get_bvecs(struct iov_iter *iter, size_t maxsize,\n--\nfs/ceph/file.c-128-\nfs/ceph/file.c:129:\t\t\tbvec_set_page(\u0026bvecs[bvec_idx], pages[idx], len, start);\nfs/ceph/file.c-130-\t\t\tbytes -= len;\n--\nfs/coredump.c=1270=static int dump_emit_page(struct coredump_params *cprm, struct page *page)\n--\nfs/coredump.c-1290-\tpos = file-\u003ef_pos;\nfs/coredump.c:1291:\tbvec_set_page(\u0026bvec, page, PAGE_SIZE, 0);\nfs/coredump.c-1292-\tiov_iter_bvec(\u0026iter, ITER_SOURCE, \u0026bvec, 1, PAGE_SIZE);\n--\nfs/dlm/lowcomms.c=1365=static int send_to_sock(struct connection *con)\n--\nfs/dlm/lowcomms.c-1386-\nfs/dlm/lowcomms.c:1387:\tbvec_set_page(\u0026bvec, e-\u003epage, len, offset);\nfs/dlm/lowcomms.c-1388-\tiov_iter_bvec(\u0026msg.msg_iter, ITER_SOURCE, \u0026bvec, 1, len);\n--\nfs/netfs/bvecq.c=162=struct bvecq *bvecq_alloc_buffer2(size_t size, unsigned int pre_slots, gfp_t gfp,\n--\nfs/netfs/bvecq.c-215-\t\t\tset_page_count(pages[i], 1);\nfs/netfs/bvecq.c:216:\t\t\tbvec_set_page(\u0026p-\u003ebv[slot++], pages[i], PAGE_SIZE, 0);\nfs/netfs/bvecq.c-217-\t\t}\n--\nfs/netfs/bvecq.c=296=int bvecq_expand_buffer(struct bvecq **_buffer, size_t *_cur_size, size_t size, gfp_t gfp)\n--\nfs/netfs/bvecq.c-338-\nfs/netfs/bvecq.c:339:\t\tbvec_set_page(\u0026tail-\u003ebv[tail-\u003enr_slots], page, PAGE_SIZE \u003c\u003c order, 0);\nfs/netfs/bvecq.c-340-\t\t*_cur_size += PAGE_SIZE \u003c\u003c order;\n--\nfs/netfs/iterator.c=36=ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,\n--\nfs/netfs/iterator.c-94-\t\t\tlen = ret \u003e PAGE_SIZE ? PAGE_SIZE : ret;\nfs/netfs/iterator.c:95:\t\t\tbvec_set_page(bv + npages + i, *pages++, len - offset, offset);\nfs/netfs/iterator.c-96-\t\t\tret -= len;\n--\nfs/nfs/localio.c=469=nfs_local_iters_init(struct nfs_local_kiocb *iocb, int rw)\n--\nfs/nfs/localio.c-483-\t\tlen = min_t(size_t, total, PAGE_SIZE - base);\nfs/nfs/localio.c:484:\t\tbvec_set_page(\u0026iocb-\u003ebvec[v], *pagevec, len, base);\nfs/nfs/localio.c-485-\t\ttotal -= len;\n--\nfs/nfsd/vfs.c=1120=nfsd_direct_read(struct svc_rqst *rqstp, struct svc_fh *fhp,\n--\nfs/nfsd/vfs.c-1144-\t\tlen = min_t(size_t, total, PAGE_SIZE);\nfs/nfsd/vfs.c:1145:\t\tbvec_set_page(\u0026rqstp-\u003erq_bvec[v], *rqstp-\u003erq_next_page,\nfs/nfsd/vfs.c-1146-\t\t\t len, 0);\n--\nfs/nfsd/vfs.c=1199=__be32 nfsd_iter_read(struct svc_rqst *rqstp, struct svc_fh *fhp,\n--\nfs/nfsd/vfs.c-1233-\t\tlen = min_t(size_t, total, PAGE_SIZE - base);\nfs/nfsd/vfs.c:1234:\t\tbvec_set_page(\u0026rqstp-\u003erq_bvec[v], *rqstp-\u003erq_next_page,\nfs/nfsd/vfs.c-1235-\t\t\t len, base);\n--\nfs/splice.c=662=iter_file_splice_write(struct pipe_inode_info *pipe, struct file *out,\n--\nfs/splice.c-726-\nfs/splice.c:727:\t\t\tbvec_set_page(\u0026array[n], buf-\u003epage, this_len,\nfs/splice.c-728-\t\t\t\t buf-\u003eoffset);\n--\nfs/splice.c=795=ssize_t splice_to_socket(struct pipe_inode_info *pipe, struct file *out,\n--\nfs/splice.c-863-\nfs/splice.c:864:\t\t\tbvec_set_page(\u0026bvec[bc++], buf-\u003epage, seg, buf-\u003eoffset);\nfs/splice.c-865-\t\t\tremain -= seg;\n--\ninclude/linux/bvec.h=28=struct bio_vec {\n--\ninclude/linux/bvec.h-34-/**\ninclude/linux/bvec.h:35: * bvec_set_page - initialize a bvec based off a struct page\ninclude/linux/bvec.h-36- * @bv:\t\tbvec to initialize\n--\ninclude/linux/bvec.h-40- */\ninclude/linux/bvec.h:41:static inline void bvec_set_page(struct bio_vec *bv, struct page *page,\ninclude/linux/bvec.h-42-\t\tunsigned int len, unsigned int offset)\n--\ninclude/linux/bvec.h=56=static inline void bvec_set_folio(struct bio_vec *bv, struct folio *folio,\n--\ninclude/linux/bvec.h-61-\tWARN_ON_ONCE(len \u003e UINT_MAX);\ninclude/linux/bvec.h:62:\tbvec_set_page(bv, folio_page(folio, nr), len, offset % PAGE_SIZE);\ninclude/linux/bvec.h-63-}\n--\ninclude/linux/bvec.h=71=static inline void bvec_set_virt(struct bio_vec *bv, void *vaddr,\n--\ninclude/linux/bvec.h-73-{\ninclude/linux/bvec.h:74:\tbvec_set_page(bv, virt_to_page(vaddr), len, offset_in_page(vaddr));\ninclude/linux/bvec.h-75-}\n--\nio_uring/rsrc.c=855=static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx,\n--\nio_uring/rsrc.c-930-\t\tvec_len = min_t(size_t, size, (1UL \u003c\u003c imu-\u003efolio_shift) - off);\nio_uring/rsrc.c:931:\t\tbvec_set_page(\u0026imu-\u003ebvec[i], pages[i], vec_len, off);\nio_uring/rsrc.c-932-\t\toff = 0;\n--\nio_uring/rsrc.c=1528=static int io_vec_fill_bvec(int ddir, struct iov_iter *iter,\n--\nio_uring/rsrc.c-1569-\nio_uring/rsrc.c:1570:\t\t\tbvec_set_page(\u0026res_bvec[bvec_idx],\nio_uring/rsrc.c-1571-\t\t\t\t src_bvec-\u003ebv_page, seg_size, offset);\n--\nlib/iov_iter.c=2117=ssize_t iov_iter_extract_bvecs(struct iov_iter *iter, struct bio_vec *bv,\n--\nlib/iov_iter.c-2169-\t\tlen = get_contig_folio_len(\u0026pages[i], \u0026nr_to_add, left, offset);\nlib/iov_iter.c:2170:\t\tbvec_set_page(\u0026bv[*nr_vecs], pages[i], len, offset);\nlib/iov_iter.c-2171-\t\ti += nr_to_add;\n--\nlib/tests/kunit_iov_iter.c=237=static void __init iov_kunit_load_bvec(struct kunit *test,\n--\nlib/tests/kunit_iov_iter.c-261-\t\t} else {\nlib/tests/kunit_iov_iter.c:262:\t\t\tbvec_set_page(\u0026bvec[i], page, pr-\u003eto - pr-\u003efrom, pr-\u003efrom);\nlib/tests/kunit_iov_iter.c-263-\t\t}\n--\nlib/tests/kunit_iov_iter.c=417=static void __init iov_kunit_load_bvecq(struct kunit *test,\n--\nlib/tests/kunit_iov_iter.c-430-\t\t}\nlib/tests/kunit_iov_iter.c:431:\t\tbvec_set_page(\u0026bq-\u003ebv[bq-\u003enr_slots], pages[i], PAGE_SIZE, 0);\nlib/tests/kunit_iov_iter.c-432-\t\tbq-\u003enr_slots++;\n--\nlib/tests/kunit_iov_iter.c=1155=static void __init iov_kunit_iter_to_sg_bvec(struct kunit *test)\n--\nlib/tests/kunit_iov_iter.c-1174-\t\telse\nlib/tests/kunit_iov_iter.c:1175:\t\t\tbvec_set_page(\u0026bvec[k++], p, PAGE_SIZE, 0);\nlib/tests/kunit_iov_iter.c-1176-\t\tcan_merge = p + 1;\n--\nnet/ceph/messenger_v1.c=40=static int ceph_tcp_recvpage(struct socket *sock, struct page *page,\n--\nnet/ceph/messenger_v1.c-47-\tBUG_ON(page_offset + length \u003e PAGE_SIZE);\nnet/ceph/messenger_v1.c:48:\tbvec_set_page(\u0026bvec, page, length, page_offset);\nnet/ceph/messenger_v1.c-49-\tiov_iter_bvec(\u0026msg.msg_iter, ITER_DEST, \u0026bvec, 1, length);\n--\nnet/ceph/messenger_v1.c=80=static int ceph_tcp_sendpage(struct socket *sock, struct page *page,\n--\nnet/ceph/messenger_v1.c-99-\nnet/ceph/messenger_v1.c:100:\tbvec_set_page(\u0026bvec, page, size, offset);\nnet/ceph/messenger_v1.c-101-\tiov_iter_bvec(\u0026msg.msg_iter, ITER_SOURCE, \u0026bvec, 1, size);\n--\nnet/ceph/messenger_v2.c=142=static int do_try_sendpage(struct socket *sock, struct iov_iter *it)\n--\nnet/ceph/messenger_v2.c-152-\t\t/* iov_iter_iovec() for ITER_BVEC */\nnet/ceph/messenger_v2.c:153:\t\tbvec_set_page(\u0026bv, it-\u003ebvec-\u003ebv_page,\nnet/ceph/messenger_v2.c-154-\t\t\t min(iov_iter_count(it),\n--\nnet/ceph/messenger_v2.c=284=static void set_out_bvec_zero(struct ceph_connection *con)\n--\nnet/ceph/messenger_v2.c-288-\nnet/ceph/messenger_v2.c:289:\tbvec_set_page(\u0026con-\u003ev2.out_bvec, ceph_zero_page,\nnet/ceph/messenger_v2.c-290-\t\t min(con-\u003ev2.out_zero, (int)PAGE_SIZE), 0);\n--\nnet/ceph/messenger_v2.c=831=static void get_bvec_at(struct ceph_msg_data_cursor *cursor,\n--\nnet/ceph/messenger_v2.c-844-\tpage = ceph_msg_data_next(cursor, \u0026off, \u0026len);\nnet/ceph/messenger_v2.c:845:\tbvec_set_page(bv, page, len, off);\nnet/ceph/messenger_v2.c-846-}\n--\nnet/ceph/messenger_v2.c=2049=static void prepare_read_enc_page(struct ceph_connection *con)\n--\nnet/ceph/messenger_v2.c-2056-\nnet/ceph/messenger_v2.c:2057:\tbvec_set_page(\u0026bv, con-\u003ev2.in_enc_pages[con-\u003ev2.in_enc_i],\nnet/ceph/messenger_v2.c-2058-\t\t min(con-\u003ev2.in_enc_resid, (int)PAGE_SIZE), 0);\n--\nnet/ceph/messenger_v2.c=3211=static void queue_enc_page(struct ceph_connection *con)\n--\nnet/ceph/messenger_v2.c-3218-\nnet/ceph/messenger_v2.c:3219:\tbvec_set_page(\u0026bv, con-\u003ev2.out_enc_pages[con-\u003ev2.out_enc_i],\nnet/ceph/messenger_v2.c-3220-\t\t min(con-\u003ev2.out_enc_resid, (int)PAGE_SIZE), 0);\n--\nnet/core/skbuff.c=3314=static int __skb_send_sock(struct sock *sk, struct sk_buff *skb, int offset,\n--\nnet/core/skbuff.c-3378-\t\t\t\tmsg.msg_flags |= more_hint;\nnet/core/skbuff.c:3379:\t\t\tbvec_set_page(\u0026bvec, skb_frag_page(frag), slen,\nnet/core/skbuff.c-3380-\t\t\t\t skb_frag_off(frag) + offset);\n--\nnet/ipv4/tcp_bpf.c=91=static int tcp_bpf_push(struct sock *sk, struct sk_msg *msg, u32 apply_bytes,\n--\nnet/ipv4/tcp_bpf.c-120-\nnet/ipv4/tcp_bpf.c:121:\t\tbvec_set_page(\u0026bvec, page, size, off);\nnet/ipv4/tcp_bpf.c-122-\t\tiov_iter_bvec(\u0026msghdr.msg_iter, ITER_SOURCE, \u0026bvec, 1, size);\n--\nnet/rds/tcp_send.c=70=int rds_tcp_xmit(struct rds_connection *conn, struct rds_message *rm,\n--\nnet/rds/tcp_send.c-119-\nnet/rds/tcp_send.c:120:\t\tbvec_set_page(\u0026bvec, sg_page(\u0026rm-\u003edata.op_sg[sg]),\nnet/rds/tcp_send.c-121-\t\t\t rm-\u003edata.op_sg[sg].length - off,\n--\nnet/rxrpc/output.c=927=void rxrpc_send_response(struct rxrpc_connection *conn, struct sk_buff *response)\n--\nnet/rxrpc/output.c-949-\tfor (int i = 0; i \u003c nr_sg; i++)\nnet/rxrpc/output.c:950:\t\tbvec_set_page(\u0026bvec[i], sg_page(\u0026sg[i]), sg[i].length, sg[i].offset);\nnet/rxrpc/output.c-951-\n--\nnet/rxrpc/rxgk.c=640=static int rxgk_issue_challenge(struct rxrpc_connection *conn)\n--\nnet/rxrpc/rxgk.c-658-\nnet/rxrpc/rxgk.c:659:\tbvec_set_page(\u0026bvec[0], page, len, 0);\nnet/rxrpc/rxgk.c-660-\tiov_iter_bvec(\u0026msg.msg_iter, WRITE, bvec, 1, len);\n--\nnet/rxrpc/rxperf.c=511=static int rxperf_process_call(struct rxperf_call *call)\n--\nnet/rxrpc/rxperf.c-523-\t\tlen = umin(reply_len, PAGE_SIZE);\nnet/rxrpc/rxperf.c:524:\t\tbvec_set_page(\u0026bv, ZERO_PAGE(0), len, 0);\nnet/rxrpc/rxperf.c-525-\t\tiov_iter_bvec(\u0026msg.msg_iter, WRITE, \u0026bv, 1, len);\n--\nnet/sunrpc/svcsock.c=339=static ssize_t svc_tcp_read_msg(struct svc_rqst *rqstp, size_t buflen,\n--\nnet/sunrpc/svcsock.c-352-\tfor (i = 0, t = 0; t \u003c buflen; i++, t += PAGE_SIZE)\nnet/sunrpc/svcsock.c:353:\t\tbvec_set_page(\u0026bvec[i], rqstp-\u003erq_pages[i], PAGE_SIZE, 0);\nnet/sunrpc/svcsock.c-354-\n--\nnet/sunrpc/xdr.c=116=xdr_alloc_bvec(struct xdr_buf *buf, gfp_t gfp)\n--\nnet/sunrpc/xdr.c-124-\t\tfor (i = 0; i \u003c n; i++) {\nnet/sunrpc/xdr.c:125:\t\t\tbvec_set_page(\u0026buf-\u003ebvec[i], buf-\u003epages[i], PAGE_SIZE,\nnet/sunrpc/xdr.c-126-\t\t\t\t 0);\n--\nnet/sunrpc/xdr.c=148=int xdr_buf_to_bvec(struct bio_vec *bvec, unsigned int bvec_size,\n--\nnet/sunrpc/xdr.c-172-\t\t\t\tgoto bvec_overflow;\nnet/sunrpc/xdr.c:173:\t\t\tbvec_set_page(bvec++, *pages++, len, offset);\nnet/sunrpc/xdr.c-174-\t\t\tremaining -= len;\n--\nnet/sunrpc/xprtrdma/svc_rdma_rw.c=445=static void svc_rdma_pagelist_to_bvec(struct svc_rdma_write_info *info,\n--\nnet/sunrpc/xprtrdma/svc_rdma_rw.c-461-\t\t\t\t PAGE_SIZE - page_off);\nnet/sunrpc/xprtrdma/svc_rdma_rw.c:462:\t\tbvec_set_page(\u0026ctxt-\u003erw_bvec[bvec_idx], *page, bvec_len,\nnet/sunrpc/xprtrdma/svc_rdma_rw.c-463-\t\t\t page_off);\n--\nnet/sunrpc/xprtrdma/svc_rdma_rw.c=760=static int svc_rdma_build_read_segment(struct svc_rqst *rqstp,\n--\nnet/sunrpc/xprtrdma/svc_rdma_rw.c-785-\nnet/sunrpc/xprtrdma/svc_rdma_rw.c:786:\t\tbvec_set_page(\u0026ctxt-\u003erw_bvec[bvec_idx],\nnet/sunrpc/xprtrdma/svc_rdma_rw.c-787-\t\t\t rqstp-\u003erq_pages[head-\u003erc_curpage],\n--\nnet/tls/tls_main.c=168=int tls_push_sg(struct sock *sk,\n--\nnet/tls/tls_main.c-191-retry:\nnet/tls/tls_main.c:192:\t\tbvec_set_page(\u0026bvec, p, size, offset);\nnet/tls/tls_main.c-193-\t\tiov_iter_bvec(\u0026msg.msg_iter, ITER_SOURCE, \u0026bvec, 1, size);\n--\nnet/xfrm/espintcp.c=216=static int espintcp_sendskmsg_locked(struct sock *sk,\n--\nnet/xfrm/espintcp.c-233-\nnet/xfrm/espintcp.c:234:\t\tbvec_set_page(\u0026bvec, sg_page(sg), sg-\u003elength, sg-\u003eoffset);\nnet/xfrm/espintcp.c-235-\t\tiov_iter_bvec(\u0026msghdr.msg_iter, ITER_SOURCE, \u0026bvec, 1, sg-\u003elength);\n"
} |
| 33/3 | 2026/09/30 13:50 | llm | kmsan-evaluator |
3mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 85421 output: 35 thoughts: 3913 Thoughts: **Analyzing Potential Uninitialized Memory Reads in `iov_iter` and Related Structures** Alright, let's break down these concerns about uninitialized memory being read through `iov_iter` and related structures. My primary focus is on ensuring data integrity and preventing security vulnerabilities. The investigation started with examining the setup of `bio_vec` and `iov_iter`, key components for representing and iterating over data buffers in the kernel. Initially, I looked into the functions `bvec_set_page` and `iov_iter_bvec_queue`, confirming that all fields of `struct bio_vec` and `struct iov_iter` are properly initialized. The aim is to understand how these structs are used and whether there are any potential gaps that could lead to uninitialized memory being exposed. The function `iterate_bvecq` is central to this discussion. It iterates through the `bvecq` (bio vector queue), accessing pages and offsets. I paid close attention to the `kmap_local_page(page) + poff` line, ensuring that the memory access through `base` within the `step` function (the actual access function which uses the memory that is to be copied into/from) doesn't read uninitialized data. Then, I shifted my focus to concrete scenarios within different filesystems and drivers. The `afs_dir_iterate_contents` and `afs_dir_check` functions, reading directory blocks, appeared safe because the data should either come from the server or have been initialized by the filesystem. However, I got particularly interested in `afs_init_new_symlink`, where there could be an issue with allocating a new buffer using `memcpy`. Thankfully, the code explicitly zeroing out any remaining portion of the allocated buffer using `memset` after `memcpy` provides a safety net! This zeroing is a crucial safeguard, preventing uninitialized memory from being exposed. I confirmed the zeroing by the check in the newer code, unlike the old code. Next, I looked at the SMB/CIFS protocol implementations. Specifically, `cifs_copy_iter_to_bvecq` and its usage within functions such as `smb3_init_transform_rq`. The goal was to check for potential reads of uninitialized memory in the context of copying data from an `iov_iter` to a `bvecq`. The concern was whether a `bvecq` allocation might have extra capacity beyond the actual data size, leaving some memory uninitialized. By tracking how the `iov_iter` and `bvecq` are used and the size passed to the latter, it appears that the count value is used to ensure the memory being read is known to be initialized. I then shifted to `smbdirect_map_sges_from_bvecq`, another iov_iter mapping, which replaced the old `smbdirect_map_sges_from_folioq`. Here, it appears to be a standard iov_iter mapping helper, which means there should be no such uninitialized memory problem. Finally, I carefully inspected `lib/iov_iter.c`, to check `iter_count_bvecq_pages` and `iter_bvecq_get_pages` to understand the low-level logic, and the `want_pages_array` function which allocates the pages array. The aim was to scrutinize the page pointer manipulation within these functions. I got concerned because `iov_iter_extract_bvecq_pages` did not call `want_pages_array`, unlike the old code where a call to `want_pages_array` was present in the function `iov_iter_extract_folioq_pages`. `want_pages_array` is critical for ensuring the pages array contains valid and properly allocated entries. The absence of this call in the new code is a significant point of concern that requires further investigation. In summary, there were some concerns in the past (now fixed by `memset` for some buffer allocations) and the current code is generally careful, but I identified a potential issue in `iov_iter_extract_bvecq_pages` regarding memory allocation that needs further scrutiny. |
| 34/3 | 2026/09/30 13:54 | tool | read-file |
0mArgs:{
"File": "lib/iov_iter.c",
"FirstLine": 900,
"LineCount": 40
}Results: {
"Contents": " 900:\t\t\t\treturn ((unsigned long)i-\u003eubuf + i-\u003eiov_offset) | size;\n 901:\t\t\treturn 0;\n 902:\t\t}\n 903:\t\n 904:\t\t/* iovec and kvec have identical layouts */\n 905:\t\tif (likely(iter_is_iovec(i) || iov_iter_is_kvec(i)))\n 906:\t\t\treturn iov_iter_alignment_iovec(i);\n 907:\t\n 908:\t\tif (iov_iter_is_bvec(i))\n 909:\t\t\treturn iov_iter_alignment_bvec(i);\n 910:\t\tif (iov_iter_is_bvecq(i))\n 911:\t\t\treturn iov_iter_alignment_bvecq(i);\n 912:\t\n 913:\t\t/* With the xarray type, we're dealing with whole folios. */\n 914:\t\tif (iov_iter_is_xarray(i))\n 915:\t\t\treturn (i-\u003exarray_start + i-\u003eiov_offset) | i-\u003ecount;\n 916:\t\n 917:\t\treturn 0;\n 918:\t}\n 919:\tEXPORT_SYMBOL(iov_iter_alignment);\n 920:\t\n 921:\tunsigned long iov_iter_gap_alignment(const struct iov_iter *i)\n 922:\t{\n 923:\t\tunsigned long res = 0;\n 924:\t\tunsigned long v = 0;\n 925:\t\tsize_t size = i-\u003ecount;\n 926:\t\tunsigned k;\n 927:\t\n 928:\t\tif (iter_is_ubuf(i))\n 929:\t\t\treturn 0;\n 930:\t\n 931:\t\tif (WARN_ON(!iter_is_iovec(i)))\n 932:\t\t\treturn ~0U;\n 933:\t\n 934:\t\tfor (k = 0; k \u003c i-\u003enr_segs; k++) {\n 935:\t\t\tconst struct iovec *iov = iter_iov(i) + k;\n 936:\t\t\tif (iov-\u003eiov_len) {\n 937:\t\t\t\tunsigned long base = (unsigned long)iov-\u003eiov_base;\n 938:\t\t\t\tif (v) // if not the first one\n 939:\t\t\t\t\tres |= base | v; // this start | previous end\n"
} |
| 35/3 | 2026/09/30 13:54 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 90899 output: 35 thoughts: 24 Thoughts: Here's a summary of my thought process: **Tracing the Source of `want_pages_array` in `lib/iov_iter.c`** Okay, so I need to find the definition of `want_pages_array`. I know it's related to `lib/iov_iter.c`, which, from experience, likely deals with the I/O vector iterator functionality in the kernel. My initial thought is to just jump right into the code and start searching within that file itself. Hopefully, it's defined locally. If not, I'll need to trace its declaration through include files or figure out how it's used to pinpoint where it's being set. It's probably a data structure or macro, judging by the name. Let's dig in and see what we can uncover! I expect to understand its purpose within the context of the I/O iteration process, probably related to allocating or managing page-aligned memory for the iteration. |
| 36/3 | 2026/09/30 13:54 | tool | read-file |
0mArgs:{
"File": "lib/iov_iter.c",
"FirstLine": 945,
"LineCount": 40
}Results: {
"Contents": " 945:\t\t}\n 946:\t\treturn res;\n 947:\t}\n 948:\tEXPORT_SYMBOL(iov_iter_gap_alignment);\n 949:\t\n 950:\tstatic int want_pages_array(struct page ***res, size_t size,\n 951:\t\t\t\t size_t start, unsigned int maxpages)\n 952:\t{\n 953:\t\tunsigned int count = DIV_ROUND_UP(size + start, PAGE_SIZE);\n 954:\t\n 955:\t\tif (count \u003e maxpages)\n 956:\t\t\tcount = maxpages;\n 957:\t\tWARN_ON(!count);\t// caller should've prevented that\n 958:\t\tif (!*res) {\n 959:\t\t\t*res = kvmalloc_objs(struct page *, count);\n 960:\t\t\tif (!*res)\n 961:\t\t\t\treturn 0;\n 962:\t\t}\n 963:\t\treturn count;\n 964:\t}\n 965:\t\n 966:\t/*\n 967:\t * Count the number of virtually contiguous pages coming up next in an\n 968:\t * ITER_BVECQ iterator, up to the specified maxima.\n 969:\t */\n 970:\tstatic unsigned int iter_count_bvecq_pages(const struct iov_iter *iter,\n 971:\t\t\t\t\t\t size_t maxsize,\n 972:\t\t\t\t\t\t unsigned int maxpages)\n 973:\t{\n 974:\t\tconst struct bvecq *bvecq = iter-\u003ebvecq;\n 975:\t\tunsigned int slot = iter-\u003ebvecq_slot;\n 976:\t\tssize_t remain = umin(maxsize, iter-\u003ecount);\n 977:\t\tsize_t count = 0, offset = iter-\u003eiov_offset;\n 978:\t\n 979:\t\tdo {\n 980:\t\t\tconst struct bio_vec *bv;\n 981:\t\t\tsize_t boff, blen;\n 982:\t\n 983:\t\t\tif (slot \u003e= bvecq-\u003enr_slots) {\n 984:\t\t\t\tif (!bvecq-\u003enext) {\n"
} |
| 37/3 | 2026/09/30 13:54 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 91745 output: 24 thoughts: 335 Thoughts: **Analyzing Page Extraction in the IO Vector Iterator** Okay, let's break this down. First, `want_pages_array` handles the initial page allocation using `kvmalloc_objs` if `*res` is null, which I know is just a wrapper for `kmalloc_array` without the zero-initialization flag. It then returns the page count. Now, I'm diving into `iov_iter_extract_bvecq_pages` to understand how the IO vector iterator extracts pages. The code snippet shows that it takes an `iov_iter`, a page array, size limits, and flags as input. It utilizes `iter_count_bvecq_pages` to determine the maximum number of virtually contiguous pages to extract, constrained by the provided `maxsize` and `maxpages` values. If `iter_count_bvecq_pages` returns 0, the function returns, implying there are no pages to extract within the given constraints. Crucially, if the provided `*pages` array is NULL, the function allocates it using `kvmalloc_array` to a size determined by `maxpages` . This indicates it's handling potential re-allocation. Then, the extraction process begins. I want to see how `iov_iter_extract_bvec_pages` processes the data. I'll need to investigate that function to completely understand this page extraction mechanism. |
| 38/3 | 2026/09/30 13:55 | tool | grepper |
0mArgs:{
"Expression": "iov_iter_extract_bvec_pages"
}Results: {
"Output": "lib/iov_iter.c=1760=static ssize_t iov_iter_extract_xarray_pages(struct iov_iter *i,\n--\nlib/iov_iter.c-1814- */\nlib/iov_iter.c:1815:static ssize_t iov_iter_extract_bvec_pages(struct iov_iter *i,\nlib/iov_iter.c-1816-\t\t\t\t\t struct page ***pages, size_t maxsize,\n--\nlib/iov_iter.c=2031=ssize_t iov_iter_extract_pages(struct iov_iter *i,\n--\nlib/iov_iter.c-2050-\tif (iov_iter_is_bvec(i))\nlib/iov_iter.c:2051:\t\treturn iov_iter_extract_bvec_pages(i, pages, maxsize,\nlib/iov_iter.c-2052-\t\t\t\t\t\t maxpages, extraction_flags,\n"
} |
| 39/3 | 2026/09/30 13:55 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 92659 output: 36 thoughts: 26 Thoughts: Okay, let's dive into this. Here's my take on understanding this function: **Analyzing `iov_iter_extract_bvec_pages`** Right, so I'm focusing on `iov_iter_extract_bvec_pages` within `lib/iov_iter.c`. My initial assumption is this function is all about efficiently extracting data from an `iov_iter` structure and writing it into a set of `bvec` pages. The "bvec" part immediately flags memory management and direct access. I'm already anticipating some of the nuances involved, especially around page boundary alignment and potential scatter/gather operations, given the name. I need to understand how it deals with partial writes and error handling; is it atomic or does it allow for partial successes. I'll be looking to understand the function's parameters: The `iov_iter`, the destination `bvec`, how it handles the number of bytes to transfer. I want to know where it's reading from. I'll need to see how it moves the `iov_iter` forward. How does it manage the `iova` mapping? How does it handle page faults? And most importantly, is this function designed for user space or kernel space access? It's likely kernel space, and therefore what are the memory barriers and locking mechanisms it uses? I need to understand how the function addresses the different states that an `iov_iter` can be in. Specifically, are we dealing with reading from user space buffers, kernel space buffers, or a combination? And, importantly, what does this function return? Bytes extracted? Error code? I'm expecting some kind of value that indicates how much was successfully copied, or a negative error code if things went south. Okay, let's see what's actually in the code. |
| 40/3 | 2026/09/30 13:55 | tool | read-file |
0mArgs:{
"File": "lib/iov_iter.c",
"FirstLine": 1815,
"LineCount": 45
}Results: {
"Contents": "1815:\tstatic ssize_t iov_iter_extract_bvec_pages(struct iov_iter *i,\n1816:\t\t\t\t\t\t struct page ***pages, size_t maxsize,\n1817:\t\t\t\t\t\t unsigned int maxpages,\n1818:\t\t\t\t\t\t iov_iter_extraction_t extraction_flags,\n1819:\t\t\t\t\t\t size_t *offset0)\n1820:\t{\n1821:\t\tsize_t skip = i-\u003eiov_offset, size = 0;\n1822:\t\tstruct bvec_iter bi;\n1823:\t\tint k = 0;\n1824:\t\n1825:\t\tif (i-\u003enr_segs == 0)\n1826:\t\t\treturn 0;\n1827:\t\n1828:\t\tif (i-\u003eiov_offset == i-\u003ebvec-\u003ebv_len) {\n1829:\t\t\ti-\u003eiov_offset = 0;\n1830:\t\t\ti-\u003enr_segs--;\n1831:\t\t\ti-\u003ebvec++;\n1832:\t\t\tskip = 0;\n1833:\t\t}\n1834:\t\tbi.bi_idx = 0;\n1835:\t\tbi.bi_size = maxsize;\n1836:\t\tbi.bi_offset = skip;\n1837:\t\n1838:\t\tmaxpages = want_pages_array(pages, maxsize, skip, maxpages);\n1839:\t\tif (!maxpages)\n1840:\t\t\treturn -ENOMEM;\n1841:\t\n1842:\t\twhile (bi.bi_size \u0026\u0026 bi.bi_idx \u003c i-\u003enr_segs) {\n1843:\t\t\tstruct bio_vec bv = bvec_iter_bvec(i-\u003ebvec, bi);\n1844:\t\n1845:\t\t\t/*\n1846:\t\t\t * The iov_iter_extract_pages interface only allows an offset\n1847:\t\t\t * into the first page. Break out of the loop if we see an\n1848:\t\t\t * offset into subsequent pages, the caller will have to call\n1849:\t\t\t * iov_iter_extract_pages again for the reminder.\n1850:\t\t\t */\n1851:\t\t\tif (k) {\n1852:\t\t\t\tif (bv.bv_offset)\n1853:\t\t\t\t\tbreak;\n1854:\t\t\t} else {\n1855:\t\t\t\t*offset0 = bv.bv_offset;\n1856:\t\t\t}\n1857:\t\n1858:\t\t\t(*pages)[k++] = bv.bv_page;\n1859:\t\t\tsize += bv.bv_len;\n"
} |
| 41/3 | 2026/09/30 13:55 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 93791 output: 21 thoughts: 45 Thoughts: Here's my analysis of the situation: **Investigating the Usage of `iov_iter_extract_pages`** Okay, let's dive in. I need to understand how `iov_iter_extract_pages` is actually being used within the system. My first thought is: *who* is calling this function? I'm going to start by getting a clear picture of the call graph. Time to reach for the trusty `grep` command. Let's search the codebase for every instance of `iov_iter_extract_pages`. That'll give me the raw data I need to begin building a mental model of its integration. I'll filter the results, focusing on the more relevant matches. This is the starting point to understand the interactions. |
| 42/3 | 2026/09/30 13:55 | tool | grepper |
0mArgs:{
"Expression": "iov_iter_extract_pages"
}Results: {
"Output": "block/bio-integrity.c=367=int bio_integrity_map_user(struct bio *bio, struct iov_iter *iter)\n--\nblock/bio-integrity.c-398-\nblock/bio-integrity.c:399:\tret = iov_iter_extract_pages(iter, \u0026pages, bytes, nr_vecs,\nblock/bio-integrity.c-400-\t\t\t\t\textraction_flags, \u0026offset);\n--\ndrivers/tee/tee_shm.c=401=register_shm_helper(struct tee_context *ctx, struct iov_iter *iter, u32 flags,\n--\ndrivers/tee/tee_shm.c-446-\ndrivers/tee/tee_shm.c:447:\tlen = iov_iter_extract_pages(iter, \u0026shm-\u003epages, LONG_MAX, num_pages, 0,\ndrivers/tee/tee_shm.c-448-\t\t\t\t \u0026off);\n--\nfs/direct-io.c-43-/*\nfs/direct-io.c:44: * How many user pages to map in one call to iov_iter_extract_pages(). This\nfs/direct-io.c-45- * determines the size of a structure in the slab cache\n--\nfs/direct-io.c=116=struct dio {\n--\nfs/direct-io.c-128-\tspinlock_t bio_lock;\t\t/* protects BIO fields below */\nfs/direct-io.c:129:\tint page_errors;\t\t/* err from iov_iter_extract_pages() */\nfs/direct-io.c-130-\tint is_async;\t\t\t/* is IO async ? */\n--\nfs/direct-io.c=166=static inline int dio_refill_pages(struct dio *dio, struct dio_submit *sdio)\n--\nfs/direct-io.c-171-\nfs/direct-io.c:172:\tret = iov_iter_extract_pages(sdio-\u003eiter, \u0026pages, LONG_MAX,\nfs/direct-io.c-173-\t\t\t\t DIO_PAGES, 0, \u0026sdio-\u003efrom);\n--\nfs/direct-io.c-202- * Get another userspace page. Returns an ERR_PTR on error. Pages are\nfs/direct-io.c:203: * buffered inside the dio so that we can call iov_iter_extract_pages()\nfs/direct-io.c-204- * against a decent number of pages, less frequently. To provide nicer use of\n--\nfs/fuse/file.c=1559=static int fuse_get_user_pages(struct fuse_args_pages *ap, struct iov_iter *ii,\n--\nfs/fuse/file.c-1570-\t * However if the implementation of fuse_conn requires pages instead of\nfs/fuse/file.c:1571:\t * pointer (e.g., virtio-fs), use iov_iter_extract_pages() instead.\nfs/fuse/file.c-1572-\t */\n--\nfs/fuse/file.c-1596-\t * Until there is support for iov_iter_extract_folios(), we have to\nfs/fuse/file.c:1597:\t * manually extract pages using iov_iter_extract_pages() and then\nfs/fuse/file.c-1598-\t * copy that to a folios array.\n--\nfs/fuse/file.c-1609-\nfs/fuse/file.c:1610:\t\tret = iov_iter_extract_pages(ii, \u0026pages,\nfs/fuse/file.c-1611-\t\t\t\t\t *nbytesp - nbytes,\n--\nfs/netfs/iterator.c=36=ssize_t netfs_extract_user_iter(struct iov_iter *orig, size_t orig_len,\n--\nfs/netfs/iterator.c-66-\twhile (count \u0026\u0026 npages \u003c max_pages) {\nfs/netfs/iterator.c:67:\t\tret = iov_iter_extract_pages(orig, \u0026pages, count,\nfs/netfs/iterator.c-68-\t\t\t\t\t max_pages - npages, extraction_flags,\n--\ninclude/linux/uio.h=371=static inline void iov_iter_ubuf(struct iov_iter *i, unsigned int direction,\n--\ninclude/linux/uio.h-386-\ninclude/linux/uio.h:387:ssize_t iov_iter_extract_pages(struct iov_iter *i, struct page ***pages,\ninclude/linux/uio.h-388-\t\t\t size_t maxsize, unsigned int maxpages,\n--\nlib/iov_iter.c=1815=static ssize_t iov_iter_extract_bvec_pages(struct iov_iter *i,\n--\nlib/iov_iter.c-1845-\t\t/*\nlib/iov_iter.c:1846:\t\t * The iov_iter_extract_pages interface only allows an offset\nlib/iov_iter.c-1847-\t\t * into the first page. Break out of the loop if we see an\nlib/iov_iter.c-1848-\t\t * offset into subsequent pages, the caller will have to call\nlib/iov_iter.c:1849:\t\t * iov_iter_extract_pages again for the reminder.\nlib/iov_iter.c-1850-\t\t */\n--\nlib/iov_iter.c-1867-\t\t * The caller will handle this with another call to\nlib/iov_iter.c:1868:\t\t * iov_iter_extract_pages.\nlib/iov_iter.c-1869-\t\t */\n--\nlib/iov_iter.c=1948=static ssize_t iov_iter_extract_user_pages(struct iov_iter *i,\n--\nlib/iov_iter.c-1988-/**\nlib/iov_iter.c:1989: * iov_iter_extract_pages - Extract a list of contiguous pages from an iterator\nlib/iov_iter.c-1990- * @i: The iterator to extract from\n--\nlib/iov_iter.c-2030- */\nlib/iov_iter.c:2031:ssize_t iov_iter_extract_pages(struct iov_iter *i,\nlib/iov_iter.c-2032-\t\t\t struct page ***pages,\n--\nlib/iov_iter.c-2063-}\nlib/iov_iter.c:2064:EXPORT_SYMBOL_GPL(iov_iter_extract_pages);\nlib/iov_iter.c-2065-\nlib/iov_iter.c=2066=static unsigned int get_contig_folio_len(struct page **pages,\n--\nlib/iov_iter.c-2109- *\nlib/iov_iter.c:2110: * Like iov_iter_extract_pages(), but returns physically contiguous ranges\nlib/iov_iter.c-2111- * contained in a single folio as a single bvec instead of multiple entries.\n--\nlib/iov_iter.c=2117=ssize_t iov_iter_extract_bvecs(struct iov_iter *iter, struct bio_vec *bv,\n--\nlib/iov_iter.c-2154-\nlib/iov_iter.c:2155:\tsize = iov_iter_extract_pages(iter, \u0026pages, max_size, entries_left,\nlib/iov_iter.c-2156-\t\t\textraction_flags, \u0026offset);\n--\nlib/scatterlist.c=1113=static ssize_t extract_user_to_sg(struct iov_iter *iter,\n--\nlib/scatterlist.c-1129-\tdo {\nlib/scatterlist.c:1130:\t\tres = iov_iter_extract_pages(iter, \u0026pages, maxsize, sg_max,\nlib/scatterlist.c-1131-\t\t\t\t\t extraction_flags, \u0026off);\n--\nlib/tests/kunit_iov_iter.c=722=static void __init iov_kunit_extract_pages_kvec(struct kunit *test)\n--\nlib/tests/kunit_iov_iter.c-749-\nlib/tests/kunit_iov_iter.c:750:\t\tlen = iov_iter_extract_pages(\u0026iter, \u0026pages, 100 * 1024,\nlib/tests/kunit_iov_iter.c-751-\t\t\t\t\t ARRAY_SIZE(pagelist), 0, \u0026offset0);\n--\nlib/tests/kunit_iov_iter.c=802=static void __init iov_kunit_extract_pages_bvec(struct kunit *test)\n--\nlib/tests/kunit_iov_iter.c-827-\nlib/tests/kunit_iov_iter.c:828:\t\tlen = iov_iter_extract_pages(\u0026iter, \u0026pages, 100 * 1024,\nlib/tests/kunit_iov_iter.c-829-\t\t\t\t\t ARRAY_SIZE(pagelist), 0, \u0026offset0);\n--\nlib/tests/kunit_iov_iter.c=880=static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)\n--\nlib/tests/kunit_iov_iter.c-911-\nlib/tests/kunit_iov_iter.c:912:\t\t\tlen = iov_iter_extract_pages(\u0026iter, \u0026pages, 100 * 1024,\nlib/tests/kunit_iov_iter.c-913-\t\t\t\t\t\t ARRAY_SIZE(pagelist), 0, \u0026offset0);\n--\nlib/tests/kunit_iov_iter.c=959=static void __init iov_kunit_extract_pages_xarray(struct kunit *test)\n--\nlib/tests/kunit_iov_iter.c-989-\nlib/tests/kunit_iov_iter.c:990:\t\t\tlen = iov_iter_extract_pages(\u0026iter, \u0026pages, 100 * 1024,\nlib/tests/kunit_iov_iter.c-991-\t\t\t\t\t\t ARRAY_SIZE(pagelist), 0, \u0026offset0);\n--\nnet/core/skbuff.c=7437=ssize_t skb_splice_from_iter(struct sk_buff *skb, struct iov_iter *iter,\n--\nnet/core/skbuff.c-7456-\nnet/core/skbuff.c:7457:\t\tlen = iov_iter_extract_pages(iter, \u0026ppages, maxsize, nr, 0, \u0026off);\nnet/core/skbuff.c-7458-\t\tif (len \u003c= 0) {\n--\nnet/rds/info.c=166=int rds_info_getsockopt(struct socket *sock, int optname, sockopt_t *opt)\n--\nnet/rds/info.c-201-\t * Preallocate the page array and pass it in so that\nnet/rds/info.c:202:\t * iov_iter_extract_pages() fills it in place rather than allocating\nnet/rds/info.c-203-\t * one for us. Handing it a non-NULL array keeps ownership of the\n--\nnet/rds/info.c-213-\nnet/rds/info.c:214:\tret = iov_iter_extract_pages(\u0026opt-\u003eiter_out, \u0026pages, len, npages,\nnet/rds/info.c-215-\t\t\t\t 0, \u0026offset0);\n--\nnet/rds/info.c-258-\t/*\nnet/rds/info.c:259:\t * iov_iter_extract_pages() pins only user-backed (ubuf) iters;\nnet/rds/info.c-260-\t * iov_iter_extract_will_pin() reports whether an unpin is owed here.\n--\nnet/tls/tls_device.c=421=static int tls_push_data(struct sock *sk,\n--\nnet/tls/tls_device.c-500-\nnet/tls/tls_device.c:501:\t\t\trc = iov_iter_extract_pages(iter, \u0026pages,\nnet/tls/tls_device.c-502-\t\t\t\t\t\t copy, 1, 0, \u0026off);\n--\nnet/tls/tls_sw.c=735=static int tls_sw_sendmsg_splice(struct sock *sk, struct msghdr *msg,\n--\nnet/tls/tls_sw.c-744-\nnet/tls/tls_sw.c:745:\t\tpart = iov_iter_extract_pages(\u0026msg-\u003emsg_iter, \u0026pages,\nnet/tls/tls_sw.c-746-\t\t\t\t\t try_to_copy, 1, 0, \u0026off);\n--\ntools/testing/selftests/net/rds/getsockopt.c-11- * - RDS_INFO_*, which pins the userspace buffer with\ntools/testing/selftests/net/rds/getsockopt.c:12: * iov_iter_extract_pages() (including a non-zero starting page offset)\ntools/testing/selftests/net/rds/getsockopt.c-13- * and lets the info producers memcpy the snapshot in under a spinlock.\n--\ntools/testing/selftests/net/rds/getsockopt.c=111=TEST_F(rds, info_counters_probe)\n--\ntools/testing/selftests/net/rds/getsockopt.c-124- * A real snapshot into an unaligned userspace buffer exercises the\ntools/testing/selftests/net/rds/getsockopt.c:125: * iov_iter_extract_pages() path, including the non-zero offset0 handling\ntools/testing/selftests/net/rds/getsockopt.c-126- * that the patch reworked. Place the buffer at a non-page-aligned address\n"
} |
| 43/3 | 2026/09/30 13:55 | llm | kmsan-evaluator |
0mModel:gemini-3.8-flash Args: null Results: null Tokens: input: 97558 output: 408 thoughts: 1973 Thoughts: **KMSAN Assessment: `NeedsKMSAN = false`**
Alright, let's break this down. My thought process is as follows: I need to determine if this patch, which introduces or modifies `bvecq` (a segmented queue of `bio_vec`) across various kernel subsystems, necessitates setting `NeedsKMSAN` to true or false. The guiding rules are clear, and I need to apply them methodically.
First, I examined where `iov_iter_extract_pages` is used, specifically checking the usage of `ITER_BVECQ`. I see that it is exclusively used for internal kernel buffers, like those used in netfs, smb, and afs, not userspace. No user-space code creates `ITER_BVECQ` directly, which means the initial potential for uninitialized data exposure to user space is zero.
Next, I went through each of the four KMSAN rules.
1. **Kernel structures to userspace:** Are any kernel structures containing uninitialized fields being sent to user space? I confirmed that there is no data being sent to user space via `copy_to_user`, `put_user`, netlink, ioctl, socket options, or BPF. No structure or padding is copied to user space, therefore rule 1 is irrelevant.
2. **Conditional logic based on uninitialized variables:** I need to check all new variables and struct fields introduced by the `bvecq` implementation.
* In `bvecq_alloc_one`, the `struct bvecq` is initialized using a C99 compound literal. All omitted fields are initialized to zero by the compiler.
* In `iov_iter_bvec_queue`, the `struct iov_iter` is initialized using a compound literal, with all omitted fields set to zero.
* In `bvecq_alloc_buffer2`, `memset` zeroes the `bv` array, ensuring that all data in the allocated slots is initialized.
* `bvec_set_folio` and `bvec_set_page` initializes page, len, and offset.
* I see `kzalloc` is used.
So, all fields are properly initialized.
3. **Uninitialized fields in complex data structures:** The `struct bvecq` itself is allocated in `bvecq_alloc_one`, where the struct members are initialized via compound literal. The `bv[]` array is cleared with `memset` in `bvecq_alloc_buffer2` before use. The `nr_slots` member is updated only after calling `bvec_set_folio`/`bvec_set_page`, and iteration always stops at `nr_slots`. Therefore no uninitialized data is ever read.
4. **Bounds checks and uninitialized bytes in existing buffers:** I look for any logic that might permit accessing uninitialized bytes due to bounds issues or length manipulations. The patch has `afs_init_new_symlink`, which includes a `memset` to zero trailing bytes, explicitly preventing information leaks. Moreover, other parts of the patch only replace `folio_queue` with `bvecq`, where the pages/folios were already allocated and filled by network/filesystem read operations or user writes.
Considering these checks, I have concluded that there is *no* chance for uninitialized memory use or information leaks.
Now, let's consider the sanitizer applicability. If there are any pointer errors (off-by-one errors or indexing errors), the errors would manifest as out-of-bounds accesses. Similarly, if `bvecq_put` had a reference count or use-after-free bug, these would also be detected as out-of-bounds memory accesses or use-after-free bugs. KASAN is ideally suited for these problems! KMSAN is specifically for finding uninitialized memory reads or information leaks to user-space, which aren't present here.
Therefore, after this thorough examination, `NeedsKMSAN` should be false. This reasoning is well-explained and should provide the complete reasoning for setting the boolean to `false`.
|
| 44/3 | 2026/09/30 13:55 | tool | set-results |
0mArgs:{
"KMSANReasoning": "The patch series replaces the `folio_queue` mechanism with `bvecq` (segmented queue of `bio_vec` structures) across netfs, AFS, CIFS/SMB, iov_iter, and scatterlist.\n\nEvaluating against KMSAN criteria:\n1. Kernel-to-user info-leaks: No kernel structures or padded structs are exposed or copied to user space. In fact, `afs_init_new_symlink()` explicitly adds a `memset` to zero trailing page data after copying symlink content.\n2. Uninitialized memory reads / branching: `struct bvecq` is allocated in `bvecq_alloc_one()` and its fields are completely initialized using a C struct literal assignment (`*bq = (struct bvecq){...}`). In `bvecq_alloc_buffer2()`, the slot array `p-\u003ebv` is explicitly cleared with `memset()`. Slots are populated using `bvec_set_page()` or `bvec_set_folio()` (which initialize all `struct bio_vec` fields) before `nr_slots` is updated via release barrier. All iterators and consumers across netfs, smb, scatterlist, and iov_iter strictly bound accesses by `nr_slots`.\n3. KASAN vs KMSAN applicability: The risks in this patch series involve buffer boundaries, pointer arithmetic (such as reusing the tail of `bvecq-\u003ebv` for `struct page **` during bulk loads), and reference counting (`bvecq_put`/`bvecq_get`). These failure modes (out-of-bounds accesses, use-after-free, double-free) are properly detected by standard KASAN and memory debugging checks. There are no uninitialized memory reads or information leaks requiring a dedicated KMSAN session.",
"NeedsKMSAN": false
}Results: {
"KMSANReasoning": "The patch series replaces the `folio_queue` mechanism with `bvecq` (segmented queue of `bio_vec` structures) across netfs, AFS, CIFS/SMB, iov_iter, and scatterlist.\n\nEvaluating against KMSAN criteria:\n1. Kernel-to-user info-leaks: No kernel structures or padded structs are exposed or copied to user space. In fact, `afs_init_new_symlink()` explicitly adds a `memset` to zero trailing page data after copying symlink content.\n2. Uninitialized memory reads / branching: `struct bvecq` is allocated in `bvecq_alloc_one()` and its fields are completely initialized using a C struct literal assignment (`*bq = (struct bvecq){...}`). In `bvecq_alloc_buffer2()`, the slot array `p-\u003ebv` is explicitly cleared with `memset()`. Slots are populated using `bvec_set_page()` or `bvec_set_folio()` (which initialize all `struct bio_vec` fields) before `nr_slots` is updated via release barrier. All iterators and consumers across netfs, smb, scatterlist, and iov_iter strictly bound accesses by `nr_slots`.\n3. KASAN vs KMSAN applicability: The risks in this patch series involve buffer boundaries, pointer arithmetic (such as reusing the tail of `bvecq-\u003ebv` for `struct page **` during bulk loads), and reference counting (`bvecq_put`/`bvecq_get`). These failure modes (out-of-bounds accesses, use-after-free, double-free) are properly detected by standard KASAN and memory debugging checks. There are no uninitialized memory reads or information leaks requiring a dedicated KMSAN session.",
"NeedsKMSAN": false
} |
| Total Calls | Total Tokens | Avg Tokens | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|
| Total Calls | Total Duration (Seconds) | Avg Duration (Seconds) |
|---|