rmap: add exclusive page to private anon_vma on swapin

On swapin it is fairly common for a page to be owned exclusively by one
process.  In that case we want to add the page to the anon_vma of that
process's VMA, instead of to the root anon_vma.

This will reduce the amount of rmap searching that the swapout code needs
to do.

Signed-off-by: Rik van Riel <riel@redhat.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 3 insertions, 1 deletions

diff --git a/mm/memory.c b/mm/memory.c
index 6b0c37dcfd16..6bc039486e9f 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2628,6 +2628,7 @@ static int do_swap_page(struct mm_struct *mm, struct vm_area_struct *vma,
         swp_entry_t entry;
         pte_t pte;
         struct mem_cgroup *ptr = NULL;
+        int exclusive = 0;
         int ret = 0;
 
         if (!pte_unmap_same(mm, pmd, page_table, orig_pte))
@@ -2722,10 +2723,11 @@ static int do_swap_page(struct mm_struct *mm, struct vm_area_struct *vma,
         if ((flags & FAULT_FLAG_WRITE) && reuse_swap_page(page)) {
                 pte = maybe_mkwrite(pte_mkdirty(pte), vma);
                 flags &= ~FAULT_FLAG_WRITE;
+                exclusive = 1;
         }
         flush_icache_page(vma, page);
         set_pte_at(mm, address, page_table, pte);
-        page_add_anon_rmap(page, vma, address);
+        do_page_add_anon_rmap(page, vma, address, exclusive);
         /* It's better to call commit-charge after rmap is established */
         mem_cgroup_commit_charge_swapin(page, ptr);