fs: Fix page cache inconsistency when mixing buffered and AIO DIO

Currently when mixing buffered reads and asynchronous direct writes it
is possible to end up with the situation where we have stale data in the
page cache while the new data is already written to disk. This is
permanent until the affected pages are flushed away. Despite the fact
that mixing buffered and direct IO is ill-advised it does pose a thread
for a data integrity, is unexpected and should be fixed.

Fix this by deferring completion of asynchronous direct writes to a
process context in the case that there are mapped pages to be found in
the inode. Later before the completion in dio_complete() invalidate
the pages in question. This ensures that after the completion the pages
in the written area are either unmapped, or populated with up-to-date
data. Also do the same for the iomap case which uses
iomap_dio_complete() instead.

This has a side effect of deferring the completion to a process context
for every AIO DIO that happens on inode that has pages mapped. However
since the consensus is that this is ill-advised practice the performance
implication should not be a problem.

This was based on proposal from Jeff Moyer, thanks!

Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Jeff Moyer <jmoyer@redhat.com>
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

1 files changed, 16 insertions, 13 deletions

diff --git a/fs/iomap.c b/fs/iomap.c
index 269b24a01f32..8194d30bdca0 100644
--- a/fs/iomap.c
+++ b/fs/iomap.c
@@ -713,8 +713,24 @@ struct iomap_dio {
 static ssize_t iomap_dio_complete(struct iomap_dio *dio)
 {
         struct kiocb *iocb = dio->iocb;
+        struct inode *inode = file_inode(iocb->ki_filp);
         ssize_t ret;
 
+        /*
+         * Try again to invalidate clean pages which might have been cached by
+         * non-direct readahead, or faulted in by get_user_pages() if the source
+         * of the write was an mmap'ed region of the file we're writing.  Either
+         * one is a pretty crazy thing to do, so we don't support it 100%.  If
+         * this invalidation fails, tough, the write still worked...
+         */
+        if (!dio->error &&
+            (dio->flags & IOMAP_DIO_WRITE) && inode->i_mapping->nrpages) {
+                ret = invalidate_inode_pages2_range(inode->i_mapping,
+                                iocb->ki_pos >> PAGE_SHIFT,
+                                (iocb->ki_pos + dio->size - 1) >> PAGE_SHIFT);
+                WARN_ON_ONCE(ret);
+        }
+
         if (dio->end_io) {
                 ret = dio->end_io(iocb,
                                 dio->error ? dio->error : dio->size,
@@ -1042,19 +1058,6 @@ iomap_dio_rw(struct kiocb *iocb, struct iov_iter *iter,
 
         ret = iomap_dio_complete(dio);
 
-        /*
-         * Try again to invalidate clean pages which might have been cached by
-         * non-direct readahead, or faulted in by get_user_pages() if the source
-         * of the write was an mmap'ed region of the file we're writing.  Either
-         * one is a pretty crazy thing to do, so we don't support it 100%.  If
-         * this invalidation fails, tough, the write still worked...
-         */
-        if (iov_iter_rw(iter) == WRITE) {
-                int err = invalidate_inode_pages2_range(mapping,
-                                start >> PAGE_SHIFT, end >> PAGE_SHIFT);
-                WARN_ON_ONCE(err);
-        }
-
         return ret;
 
 out_free_dio: