From: Zhang Yi The block range calculation in ifs_clear_range_dirty() is incorrect when partially clearing a range in a folio. We cannot clear the dirty bit of the first block or the last block if the start or end offset is not blocksize-aligned. This has not yet caused any issues since we always clear a whole folio in iomap_writeback_folio(). Fix this by rounding up the first block to blocksize alignment, and calculate the last block by rounding down (using truncation). Correct the nr_blks calculation accordingly. Fixes: 4ce02c679722 ("iomap: Add per-block dirty state tracking to improve performance") Signed-off-by: Zhang Yi Reviewed-by: Joanne Koong Reviewed-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- fs/iomap/buffered-io.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index d55b936e6986..48878bc4ff62 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -176,13 +176,17 @@ static void ifs_clear_range_dirty(struct folio *folio, { struct inode *inode = folio->mapping->host; unsigned int blks_per_folio = i_blocks_per_folio(inode, folio); - unsigned int first_blk = (off >> inode->i_blkbits); - unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; - unsigned int nr_blks = last_blk - first_blk + 1; + unsigned int first_blk = round_up(off, i_blocksize(inode)) >> + inode->i_blkbits; + unsigned int last_blk = (off + len) >> inode->i_blkbits; unsigned long flags; + if (first_blk >= last_blk) + return; + spin_lock_irqsave(&ifs->state_lock, flags); - bitmap_clear(ifs->state, first_blk + blks_per_folio, nr_blks); + bitmap_clear(ifs->state, first_blk + blks_per_folio, + last_blk - first_blk); spin_unlock_irqrestore(&ifs->state_lock, flags); } -- 2.52.0 From: Zhang Yi Current iomap_invalidate_folio() can only invalidate an entire folio. If we truncate a partial folio on a filesystem where the block size is smaller than the folio size, it will leave behind dirty bits for the truncated or punched blocks. During the write-back process, it will attempt to map the invalid hole range. Fortunately, this has not caused any real problems so far because the ->writeback_range() function corrects the length. However, the implementation of FALLOC_FL_ZERO_RANGE in ext4 depends on the support for invalidating partial folios. When ext4 partially zeroes out a dirty and unwritten folio, it does not perform a flush first like XFS. Therefore, if the dirty bits of the corresponding area cannot be cleared, the zeroed area after writeback remains in the written state rather than reverting to the unwritten state. Fix this by supporting invalidation of partial folios. Signed-off-by: Zhang Yi Reviewed-by: "Darrick J. Wong" Reviewed-by: Joanne Koong Reviewed-by: Christoph Hellwig --- fs/iomap/buffered-io.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 48878bc4ff62..aeeed6316cf9 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -761,6 +761,8 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len) WARN_ON_ONCE(folio_test_writeback(folio)); folio_cancel_dirty(folio); ifs_free(folio); + } else { + iomap_clear_range_dirty(folio, offset, len); } } EXPORT_SYMBOL_GPL(iomap_invalidate_folio); -- 2.52.0 From: Zhang Yi The did_zero output parameter was unconditionally set after the loop, which is incorrect. It should only be set when the zeroing operation actually completes, not when IOMAP_F_STALE is set or when IOMAP_F_FOLIO_BATCH is set but !folio causes the loop to break early, or when iomap_iter_advance() returns an error. This causes did_zero to be incorrectly set when zeroing a clean unwritten extent because the loop exits early without actually zeroing any data. Fix it by using a local variable to track whether any folio was actually zeroed, and only set did_zero after the loop if zeroing happened. Fixes: 98eb8d95025b ("iomap: set did_zero to true when zeroing successfully") Signed-off-by: Zhang Yi Reviewed-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- fs/iomap/buffered-io.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index aeeed6316cf9..7f77a1f68160 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -1542,6 +1542,7 @@ static int iomap_zero_iter(struct iomap_iter *iter, bool *did_zero, const struct iomap_write_ops *write_ops) { u64 bytes = iomap_length(iter); + bool zeroed = false; int status; do { @@ -1560,6 +1561,8 @@ static int iomap_zero_iter(struct iomap_iter *iter, bool *did_zero, /* a NULL folio means we're done with a folio batch */ if (!folio) { status = iomap_iter_advance_full(iter); + if (status) + return status; break; } @@ -1570,6 +1573,7 @@ static int iomap_zero_iter(struct iomap_iter *iter, bool *did_zero, bytes); folio_zero_range(folio, offset, bytes); + zeroed = true; folio_mark_accessed(folio); ret = iomap_write_end(iter, bytes, bytes, folio); @@ -1579,10 +1583,10 @@ static int iomap_zero_iter(struct iomap_iter *iter, bool *did_zero, status = iomap_iter_advance(iter, bytes); if (status) - break; + return status; } while ((bytes = iomap_length(iter)) > 0); - if (did_zero) + if (did_zero && zeroed) *did_zero = true; return status; } -- 2.52.0 From: Zhang Yi ifs_set_range_dirty() and ifs_set_range_uptodate() compute last_blk as (off + len - 1) >> i_blkbits. When off is 0 and len is 0, the unsigned subtraction underflows to SIZE_MAX, producing a huge last_blk and nr_blks value that causes bitmap_set() to write far beyond the ifs->state allocation. Regarding ifs_set_range_uptodate(), it is temporarily safe because len cannot be passed in as 0. However, for ifs_set_range_dirty() this is reachable from __iomap_write_end(): when copy_folio_from_iter_atomic() returns 0 (e.g. user buffer fault) and the folio is already uptodate, the guard at the top of __iomap_write_end() does not trigger because !folio_test_uptodate() is false, and iomap_set_range_dirty() is called with copied == 0. Add a !len guard to both functions before the computation, so that a zero-length range is a no-op. Fixes: 4ce02c679722 ("iomap: Add per-block dirty state tracking to improve performance") Cc: # v6.6 Signed-off-by: Zhang Yi Reviewed-by: Joanne Koong Reviewed-by: "Darrick J. Wong" Reviewed-by: Christoph Hellwig --- fs/iomap/buffered-io.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 7f77a1f68160..a356efbae6b0 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -67,11 +67,13 @@ static bool ifs_set_range_uptodate(struct folio *folio, struct iomap_folio_state *ifs, size_t off, size_t len) { struct inode *inode = folio->mapping->host; - unsigned int first_blk = off >> inode->i_blkbits; - unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; - unsigned int nr_blks = last_blk - first_blk + 1; + unsigned int first_blk, last_blk; - bitmap_set(ifs->state, first_blk, nr_blks); + if (len) { + first_blk = off >> inode->i_blkbits; + last_blk = (off + len - 1) >> inode->i_blkbits; + bitmap_set(ifs->state, first_blk, last_blk - first_blk + 1); + } return ifs_is_fully_uptodate(folio, ifs); } @@ -203,13 +205,17 @@ static void ifs_set_range_dirty(struct folio *folio, { struct inode *inode = folio->mapping->host; unsigned int blks_per_folio = i_blocks_per_folio(inode, folio); - unsigned int first_blk = (off >> inode->i_blkbits); - unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; - unsigned int nr_blks = last_blk - first_blk + 1; + unsigned int first_blk, last_blk; unsigned long flags; + if (!len) + return; + + first_blk = off >> inode->i_blkbits; + last_blk = (off + len - 1) >> inode->i_blkbits; spin_lock_irqsave(&ifs->state_lock, flags); - bitmap_set(ifs->state, first_blk + blks_per_folio, nr_blks); + bitmap_set(ifs->state, first_blk + blks_per_folio, + last_blk - first_blk + 1); spin_unlock_irqrestore(&ifs->state_lock, flags); } -- 2.52.0 From: Zhang Yi The range alignment strategy differs between ifs_clear_range_dirty() and ifs_set_range_dirty(). The former rounds inwards to clear only fully-covered blocks, while the latter rounds outwards to mark any partially-touched block as dirty. Add comments to document this asymmetry in block range calculation. Suggested-by: "Darrick J. Wong" Signed-off-by: Zhang Yi Reviewed-by: Joanne Koong Reviewed-by: Christoph Hellwig --- fs/iomap/buffered-io.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index a356efbae6b0..c12237ed7cde 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -173,6 +173,13 @@ static unsigned iomap_find_dirty_range(struct folio *folio, u64 *range_start, return range_end - *range_start; } +/* + * Clear the per-block dirty bits for the range [@off, @off + @len) within a + * folio. The range is rounded inwards so that only blocks fully covered by + * the range are cleared. This is required for operations like folio + * invalidation, where we must ensure a block is fully clean before discarding + * it. + */ static void ifs_clear_range_dirty(struct folio *folio, struct iomap_folio_state *ifs, size_t off, size_t len) { @@ -200,6 +207,13 @@ static void iomap_clear_range_dirty(struct folio *folio, size_t off, size_t len) ifs_clear_range_dirty(folio, ifs, off, len); } +/* + * Set the per-block dirty bits for the range [@off, @off + @len) within a + * folio. The range is rounded outwards so that any block partially touched + * by the range is marked dirty. This ensures blocks containing even a + * single dirty byte will be included in subsequent writeback, preventing + * data loss when partial blocks are written. + */ static void ifs_set_range_dirty(struct folio *folio, struct iomap_folio_state *ifs, size_t off, size_t len) { -- 2.52.0