aboutsummaryrefslogtreecommitdiffstats
path: root/mm/migrate.c
diff options
context:
space:
mode:
authorMel Gorman <mgorman@suse.de>2013-10-07 06:29:07 -0400
committerIngo Molnar <mingo@kernel.org>2013-10-09 06:40:35 -0400
commitb795854b1fa70f6aee923ae5df74ff7afeaddcaa (patch)
treefd109d9f3778c7bc934fedb3cda2b5bfb1293375 /mm/migrate.c
parent073b5beea735c7e1970686c94ff1f3aaac790a2a (diff)
sched/numa: Set preferred NUMA node based on number of private faults
Ideally it would be possible to distinguish between NUMA hinting faults that are private to a task and those that are shared. If treated identically there is a risk that shared pages bounce between nodes depending on the order they are referenced by tasks. Ultimately what is desirable is that task private pages remain local to the task while shared pages are interleaved between sharing tasks running on different nodes to give good average performance. This is further complicated by THP as even applications that partition their data may not be partitioning on a huge page boundary. To start with, this patch assumes that multi-threaded or multi-process applications partition their data and that in general the private accesses are more important for cpu->memory locality in the general case. Also, no new infrastructure is required to treat private pages properly but interleaving for shared pages requires additional infrastructure. To detect private accesses the pid of the last accessing task is required but the storage requirements are a high. This patch borrows heavily from Ingo Molnar's patch "numa, mm, sched: Implement last-CPU+PID hash tracking" to encode some bits from the last accessing task in the page flags as well as the node information. Collisions will occur but it is better than just depending on the node information. Node information is then used to determine if a page needs to migrate. The PID information is used to detect private/shared accesses. The preferred NUMA node is selected based on where the maximum number of approximately private faults were measured. Shared faults are not taken into consideration for a few reasons. First, if there are many tasks sharing the page then they'll all move towards the same node. The node will be compute overloaded and then scheduled away later only to bounce back again. Alternatively the shared tasks would just bounce around nodes because the fault information is effectively noise. Either way accounting for shared faults the same as private faults can result in lower performance overall. The second reason is based on a hypothetical workload that has a small number of very important, heavily accessed private pages but a large shared array. The shared array would dominate the number of faults and be selected as a preferred node even though it's the wrong decision. The third reason is that multiple threads in a process will race each other to fault the shared page making the fault information unreliable. Signed-off-by: Mel Gorman <mgorman@suse.de> [ Fix complication error when !NUMA_BALANCING. ] Reviewed-by: Rik van Riel <riel@redhat.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: Johannes Weiner <hannes@cmpxchg.org> Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com> Signed-off-by: Peter Zijlstra <peterz@infradead.org> Link: http://lkml.kernel.org/r/1381141781-10992-30-git-send-email-mgorman@suse.de Signed-off-by: Ingo Molnar <mingo@kernel.org>
Diffstat (limited to 'mm/migrate.c')
-rw-r--r--mm/migrate.c4
1 files changed, 2 insertions, 2 deletions
diff --git a/mm/migrate.c b/mm/migrate.c
index fcba2f46bb80..025d1e3d2ad2 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -1498,7 +1498,7 @@ static struct page *alloc_misplaced_dst_page(struct page *page,
1498 __GFP_NOWARN) & 1498 __GFP_NOWARN) &
1499 ~GFP_IOFS, 0); 1499 ~GFP_IOFS, 0);
1500 if (newpage) 1500 if (newpage)
1501 page_nid_xchg_last(newpage, page_nid_last(page)); 1501 page_nidpid_xchg_last(newpage, page_nidpid_last(page));
1502 1502
1503 return newpage; 1503 return newpage;
1504} 1504}
@@ -1675,7 +1675,7 @@ int migrate_misplaced_transhuge_page(struct mm_struct *mm,
1675 if (!new_page) 1675 if (!new_page)
1676 goto out_fail; 1676 goto out_fail;
1677 1677
1678 page_nid_xchg_last(new_page, page_nid_last(page)); 1678 page_nidpid_xchg_last(new_page, page_nidpid_last(page));
1679 1679
1680 isolated = numamigrate_isolate_page(pgdat, page); 1680 isolated = numamigrate_isolate_page(pgdat, page);
1681 if (!isolated) { 1681 if (!isolated) {