x86: fold pda into percpu area on SMP

[ Based on original patch from Christoph Lameter and Mike Travis. ]

Currently pdas and percpu areas are allocated separately.  %gs points
to local pda and percpu area can be reached using pda->data_offset.
This patch folds pda into percpu area.

Due to strange gcc requirement, pda needs to be at the beginning of
the percpu area so that pda->stack_canary is at %gs:40.  To achieve
this, a new percpu output section macro - PERCPU_VADDR_PREALLOC() - is
added and used to reserve pda sized chunk at the start of the percpu
area.

After this change, for boot cpu, %gs first points to pda in the
data.init area and later during setup_per_cpu_areas() gets updated to
point to the actual pda.  This means that setup_per_cpu_areas() need
to reload %gs for CPU0 while clearing pda area for other cpus as cpu0
already has modified it when control reaches setup_per_cpu_areas().

This patch also removes now unnecessary get_local_pda() and its call
sites.

A lot of this patch is taken from Mike Travis' "x86_64: Fold pda into
per cpu area" patch.

Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Ingo Molnar <mingo@elte.hu>

1 files changed, 11 insertions, 4 deletions

diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S
index 2f0ab0089883..7a995d0e9f78 100644
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -245,10 +245,13 @@ ENTRY(secondary_startup_64)
 
         /* Set up %gs.
          *
-         * %gs should point to the pda.  For initial boot, make %gs point
-         * to the _boot_cpu_pda in data section.  For a secondary CPU,
-         * initial_gs should be set to its pda address before the CPU runs
-         * this code.
+         * On SMP, %gs should point to the per-cpu area.  For initial
+         * boot, make %gs point to the init data section.  For a
+         * secondary CPU,initial_gs should be set to its pda address
+         * before the CPU runs this code.
+         *
+         * On UP, initial_gs points to _boot_cpu_pda and doesn't
+         * change.
          */
         movl    $MSR_GS_BASE,%ecx
         movq    initial_gs(%rip),%rax
@@ -278,7 +281,11 @@ ENTRY(secondary_startup_64)
         ENTRY(initial_code)
         .quad   x86_64_start_kernel
         ENTRY(initial_gs)
+#ifdef CONFIG_SMP
+        .quad   __per_cpu_load
+#else
         .quad   _boot_cpu_pda
+#endif
         __FINITDATA
 
         ENTRY(stack_start)