mm: mempolicy: Let vma_merge and vma_split handle vma->vm_policy linkages

Dave Jones' system call fuzz testing tool "trinity" triggered the
following bug error with slab debugging enabled

    =============================================================================
    BUG numa_policy (Not tainted): Poison overwritten
    -----------------------------------------------------------------------------

    INFO: 0xffff880146498250-0xffff880146498250. First byte 0x6a instead of 0x6b
    INFO: Allocated in mpol_new+0xa3/0x140 age=46310 cpu=6 pid=32154
     __slab_alloc+0x3d3/0x445
     kmem_cache_alloc+0x29d/0x2b0
     mpol_new+0xa3/0x140
     sys_mbind+0x142/0x620
     system_call_fastpath+0x16/0x1b
    INFO: Freed in __mpol_put+0x27/0x30 age=46268 cpu=6 pid=32154
     __slab_free+0x2e/0x1de
     kmem_cache_free+0x25a/0x260
     __mpol_put+0x27/0x30
     remove_vma+0x68/0x90
     exit_mmap+0x118/0x140
     mmput+0x73/0x110
     exit_mm+0x108/0x130
     do_exit+0x162/0xb90
     do_group_exit+0x4f/0xc0
     sys_exit_group+0x17/0x20
     system_call_fastpath+0x16/0x1b
    INFO: Slab 0xffffea0005192600 objects=27 used=27 fp=0x          (null) flags=0x20000000004080
    INFO: Object 0xffff880146498250 @offset=592 fp=0xffff88014649b9d0

This implied a reference counting bug and the problem happened during
mbind().

mbind() applies a new memory policy to a range and uses mbind_range() to
merge existing VMAs or split them as necessary.  In the event of splits,
mpol_dup() will allocate a new struct mempolicy and maintain existing
reference counts whose rules are documented in
Documentation/vm/numa_memory_policy.txt .

The problem occurs with shared memory policies.  The vm_op->set_policy
increments the reference count if necessary and split_vma() and
vma_merge() have already handled the existing reference counts.
However, policy_vma() screws it up by replacing an existing
vma->vm_policy with one that potentially has the wrong reference count
leading to a premature free.  This patch removes the damage caused by
policy_vma().

With this patch applied Dave's trinity tool runs an mbind test for 5
minutes without error.  /proc/slabinfo reported that there are no
numa_policy or shared_policy_node objects allocated after the test
completed and the shared memory region was deleted.

Signed-off-by: Mel Gorman <mgorman@suse.de>
Cc: Dave Jones <davej@redhat.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Stephen Wilson <wilsons@start.ca>
Cc: Christoph Lameter <cl@linux.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 17 insertions, 24 deletions

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 39fd416ae14f..88f9422b92e7 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -607,27 +607,6 @@ check_range(struct mm_struct *mm, unsigned long start, unsigned long end,
         return first;
 }
 
-/* Apply policy to a single VMA */
-static int policy_vma(struct vm_area_struct *vma, struct mempolicy *new)
-{
-        int err = 0;
-        struct mempolicy *old = vma->vm_policy;
-
-        pr_debug("vma %lx-%lx/%lx vm_ops %p vm_file %p set_policy %p\n",
-                 vma->vm_start, vma->vm_end, vma->vm_pgoff,
-                 vma->vm_ops, vma->vm_file,
-                 vma->vm_ops ? vma->vm_ops->set_policy : NULL);
-
-        if (vma->vm_ops && vma->vm_ops->set_policy)
-                err = vma->vm_ops->set_policy(vma, new);
-        if (!err) {
-                mpol_get(new);
-                vma->vm_policy = new;
-                mpol_put(old);
-        }
-        return err;
-}
-
 /* Step 2: apply policy to a range and do splits. */
 static int mbind_range(struct mm_struct *mm, unsigned long start,
                        unsigned long end, struct mempolicy *new_pol)
@@ -676,9 +655,23 @@ static int mbind_range(struct mm_struct *mm, unsigned long start,
                         if (err)
                                 goto out;
                 }
-                err = policy_vma(vma, new_pol);
-                if (err)
-                        goto out;
+
+                /*
+                 * Apply policy to a single VMA. The reference counting of
+                 * policy for vma_policy linkages has already been handled by
+                 * vma_merge and split_vma as necessary. If this is a shared
+                 * policy then ->set_policy will increment the reference count
+                 * for an sp node.
+                 */
+                pr_debug("vma %lx-%lx/%lx vm_ops %p vm_file %p set_policy %p\n",
+                        vma->vm_start, vma->vm_end, vma->vm_pgoff,
+                        vma->vm_ops, vma->vm_file,
+                        vma->vm_ops ? vma->vm_ops->set_policy : NULL);
+                if (vma->vm_ops && vma->vm_ops->set_policy) {
+                        err = vma->vm_ops->set_policy(vma, new_pol);
+                        if (err)
+                                goto out;
+                }
         }
 
  out: