x86_64: Quicklist support for x86_64

This adds caching of pgds and puds, pmds, pte.  That way we can avoid costly
zeroing and initialization of special mappings in the pgd.

A second quicklist is useful to separate out PGD handling.  We can carry the
initialized pgds over to the next process needing them.

Also clean up the pgd_list handling to use regular list macros.  There is no
need anymore to avoid the lru field.

Move the add/removal of the pgds to the pgdlist into the constructor /
destructor.  That way the implementation is congruent with i386.

Signed-off-by: Christoph Lameter <clameter@sgi.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: "Luck, Tony" <tony.luck@intel.com>
Acked-by: William Lee Irwin III <wli@holomorphy.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Andi Kleen <ak@suse.de>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

3 files changed, 10 insertions, 1 deletions

diff --git a/arch/x86_64/Kconfig b/arch/x86_64/Kconfig
index fca1a85042ea..45f82ae6d389 100644
--- a/arch/x86_64/Kconfig
+++ b/arch/x86_64/Kconfig
@@ -60,6 +60,14 @@ config ZONE_DMA
         bool
         default y
 
+config QUICKLIST
+        bool
+        default y
+
+config NR_QUICK
+        int
+        default 2
+
 config ISA
         bool
 
diff --git a/arch/x86_64/kernel/process.c b/arch/x86_64/kernel/process.c
index 5909039f37aa..180f4c0fcbc4 100644
--- a/arch/x86_64/kernel/process.c
+++ b/arch/x86_64/kernel/process.c
@@ -207,6 +207,7 @@ void cpu_idle (void)
                         if (__get_cpu_var(cpu_idle_state))
                                 __get_cpu_var(cpu_idle_state) = 0;
 
+                        check_pgt_cache();
                         rmb();
                         idle = pm_idle;
                         if (!idle)
diff --git a/arch/x86_64/kernel/smp.c b/arch/x86_64/kernel/smp.c
index df4a82812adb..673a300b5944 100644
--- a/arch/x86_64/kernel/smp.c
+++ b/arch/x86_64/kernel/smp.c
@@ -241,7 +241,7 @@ void flush_tlb_mm (struct mm_struct * mm)
         }
         if (!cpus_empty(cpu_mask))
                 flush_tlb_others(cpu_mask, mm, FLUSH_ALL);
-
+        check_pgt_cache();
         preempt_enable();
 }
 EXPORT_SYMBOL(flush_tlb_mm);