[PATCH] mm: tracking shared dirty pages

Tracking of dirty pages in shared writeable mmap()s.

The idea is simple: write protect clean shared writeable pages, catch the
write-fault, make writeable and set dirty.  On page write-back clean all the
PTE dirty bits and write protect them once again.

The implementation is a tad harder, mainly because the default
backing_dev_info capabilities were too loosely maintained.  Hence it is not
enough to test the backing_dev_info for cap_account_dirty.

The current heuristic is as follows, a VMA is eligible when:
 - its shared writeable
    (vm_flags & (VM_WRITE|VM_SHARED)) == (VM_WRITE|VM_SHARED)
 - it is not a 'special' mapping
    (vm_flags & (VM_PFNMAP|VM_INSERTPAGE)) == 0
 - the backing_dev_info is cap_account_dirty
    mapping_cap_account_dirty(vma->vm_file->f_mapping)
 - f_op->mmap() didn't change the default page protection

Page from remap_pfn_range() are explicitly excluded because their COW
semantics are already horrid enough (see vm_normal_page() in do_wp_page()) and
because they don't have a backing store anyway.

mprotect() is taught about the new behaviour as well.  However it overrides
the last condition.

Cleaning the pages on write-back is done with page_mkclean() a new rmap call.
It can be called on any page, but is currently only implemented for mapped
pages, if the page is found the be of a VMA that accounts dirty pages it will
also wrprotect the PTE.

Finally, in fs/buffers.c:try_to_free_buffers(); remove clear_page_dirty() from
under ->private_lock.  This seems to be safe, since ->private_lock is used to
serialize access to the buffers, not the page itself.  This is needed because
clear_page_dirty() will call into page_mkclean() and would thereby violate
locking order.

[dhowells@redhat.com: Provide a page_mkclean() implementation for NOMMU]
Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Hugh Dickins <hugh@veritas.com>
Signed-off-by: David Howells <dhowells@redhat.com>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>

1 files changed, 65 insertions, 0 deletions

diff --git a/mm/rmap.c b/mm/rmap.c
index 40158b59729e..e2155d791d99 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -434,6 +434,71 @@ int page_referenced(struct page *page, int is_locked)
         return referenced;
 }
 
+static int page_mkclean_one(struct page *page, struct vm_area_struct *vma)
+{
+        struct mm_struct *mm = vma->vm_mm;
+        unsigned long address;
+        pte_t *pte, entry;
+        spinlock_t *ptl;
+        int ret = 0;
+
+        address = vma_address(page, vma);
+        if (address == -EFAULT)
+                goto out;
+
+        pte = page_check_address(page, mm, address, &ptl);
+        if (!pte)
+                goto out;
+
+        if (!pte_dirty(*pte) && !pte_write(*pte))
+                goto unlock;
+
+        entry = ptep_get_and_clear(mm, address, pte);
+        entry = pte_mkclean(entry);
+        entry = pte_wrprotect(entry);
+        ptep_establish(vma, address, pte, entry);
+        lazy_mmu_prot_update(entry);
+        ret = 1;
+
+unlock:
+        pte_unmap_unlock(pte, ptl);
+out:
+        return ret;
+}
+
+static int page_mkclean_file(struct address_space *mapping, struct page *page)
+{
+        pgoff_t pgoff = page->index << (PAGE_CACHE_SHIFT - PAGE_SHIFT);
+        struct vm_area_struct *vma;
+        struct prio_tree_iter iter;
+        int ret = 0;
+
+        BUG_ON(PageAnon(page));
+
+        spin_lock(&mapping->i_mmap_lock);
+        vma_prio_tree_foreach(vma, &iter, &mapping->i_mmap, pgoff, pgoff) {
+                if (vma->vm_flags & VM_SHARED)
+                        ret += page_mkclean_one(page, vma);
+        }
+        spin_unlock(&mapping->i_mmap_lock);
+        return ret;
+}
+
+int page_mkclean(struct page *page)
+{
+        int ret = 0;
+
+        BUG_ON(!PageLocked(page));
+
+        if (page_mapped(page)) {
+                struct address_space *mapping = page_mapping(page);
+                if (mapping)
+                        ret = page_mkclean_file(mapping, page);
+        }
+
+        return ret;
+}
+
 /**
  * page_set_anon_rmap - setup new anonymous rmap
  * @page:       the page to add the mapping to