page allocator: do not check NUMA node ID when the caller knows the node is valid

Callers of alloc_pages_node() can optionally specify -1 as a node to mean
"allocate from the current node".  However, a number of the callers in
fast paths know for a fact their node is valid.  To avoid a comparison and
branch, this patch adds alloc_pages_exact_node() that only checks the nid
with VM_BUG_ON().  Callers that know their node is valid are then
converted.

Signed-off-by: Mel Gorman <mel@csn.ul.ie>
Reviewed-by: Christoph Lameter <cl@linux-foundation.org>
Reviewed-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Reviewed-by: Pekka Enberg <penberg@cs.helsinki.fi>
Acked-by: Paul Mundt <lethal@linux-sh.org>	[for the SLOB NUMA bits]
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Cc: Dave Hansen <dave@linux.vnet.ibm.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 4 insertions, 4 deletions

diff --git a/kernel/profile.c b/kernel/profile.c
index 28cf26ad2d24..69911b5745eb 100644
--- a/kernel/profile.c
+++ b/kernel/profile.c
@@ -365,7 +365,7 @@ static int __cpuinit profile_cpu_callback(struct notifier_block *info,
                 node = cpu_to_node(cpu);
                 per_cpu(cpu_profile_flip, cpu) = 0;
                 if (!per_cpu(cpu_profile_hits, cpu)[1]) {
-                        page = alloc_pages_node(node,
+                        page = alloc_pages_exact_node(node,
                                         GFP_KERNEL | __GFP_ZERO,
                                         0);
                         if (!page)
@@ -373,7 +373,7 @@ static int __cpuinit profile_cpu_callback(struct notifier_block *info,
                         per_cpu(cpu_profile_hits, cpu)[1] = page_address(page);
                 }
                 if (!per_cpu(cpu_profile_hits, cpu)[0]) {
-                        page = alloc_pages_node(node,
+                        page = alloc_pages_exact_node(node,
                                         GFP_KERNEL | __GFP_ZERO,
                                         0);
                         if (!page)
@@ -564,14 +564,14 @@ static int create_hash_tables(void)
                 int node = cpu_to_node(cpu);
                 struct page *page;
 
-                page = alloc_pages_node(node,
+                page = alloc_pages_exact_node(node,
                                 GFP_KERNEL | __GFP_ZERO | GFP_THISNODE,
                                 0);
                 if (!page)
                         goto out_cleanup;
                 per_cpu(cpu_profile_hits, cpu)[1]
                                 = (struct profile_hit *)page_address(page);
-                page = alloc_pages_node(node,
+                page = alloc_pages_exact_node(node,
                                 GFP_KERNEL | __GFP_ZERO | GFP_THISNODE,
                                 0);
                 if (!page)