sched/numa: Adjust scan rate in task_numa_placement

Adjust numa_scan_period in task_numa_placement, depending on how much
useful work the numa code can do. The more local faults there are in a
given scan window the longer the period (and hence the slower the scan rate)
during the next window. If there are excessive shared faults then the scan
period will decrease with the amount of scaling depending on whether the
ratio of shared/private faults. If the preferred node changes then the
scan rate is reset to recheck if the task is properly placed.

Signed-off-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Mel Gorman <mgorman@suse.de>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1381141781-10992-59-git-send-email-mgorman@suse.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>

2 files changed, 9 insertions, 4 deletions

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 7ab4e32afe12..1be2a1f95b61 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1296,8 +1296,10 @@ int do_huge_pmd_numa_page(struct mm_struct *mm, struct vm_area_struct *vma,
         page_nid = page_to_nid(page);
         last_cpupid = page_cpupid_last(page);
         count_vm_numa_event(NUMA_HINT_FAULTS);
-        if (page_nid == this_nid)
+        if (page_nid == this_nid) {
                 count_vm_numa_event(NUMA_HINT_FAULTS_LOCAL);
+                flags |= TNF_FAULT_LOCAL;
+        }
 
         /*
          * Avoid grouping on DSO/COW pages in specific and RO pages
diff --git a/mm/memory.c b/mm/memory.c
index 823720c43ea9..1c7501f7fb1a 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3527,13 +3527,16 @@ static int do_nonlinear_fault(struct mm_struct *mm, struct vm_area_struct *vma,
 }
 
 int numa_migrate_prep(struct page *page, struct vm_area_struct *vma,
-                                unsigned long addr, int page_nid)
+                                unsigned long addr, int page_nid,
+                                int *flags)
 {
         get_page(page);
 
         count_vm_numa_event(NUMA_HINT_FAULTS);
-        if (page_nid == numa_node_id())
+        if (page_nid == numa_node_id()) {
                 count_vm_numa_event(NUMA_HINT_FAULTS_LOCAL);
+                *flags |= TNF_FAULT_LOCAL;
+        }
 
         return mpol_misplaced(page, vma, addr);
 }
@@ -3593,7 +3596,7 @@ int do_numa_page(struct mm_struct *mm, struct vm_area_struct *vma,
 
         last_cpupid = page_cpupid_last(page);
         page_nid = page_to_nid(page);
-        target_nid = numa_migrate_prep(page, vma, addr, page_nid);
+        target_nid = numa_migrate_prep(page, vma, addr, page_nid, &flags);
         pte_unmap_unlock(ptep, ptl);
         if (target_nid == -1) {
                 put_page(page);