x86: Save cr2 in NMI in case NMIs take a page fault

Avi Kivity reported that page faults in NMIs could cause havic if
the NMI preempted another page fault handler:

   The recent changes to NMI allow exceptions to take place in NMI
   handlers, but I think that a #PF (say, due to access to vmalloc space)
   is still problematic.  Consider the sequence

    #PF  (cr2 set by processor)
      NMI
        ...
        #PF (cr2 clobbered)
          do_page_fault()
          IRET
        ...
        IRET
      do_page_fault()
        address = read_cr2()

   The last line reads the overwritten cr2 value.

Originally I wrote a patch to solve this by saving the cr2 on the stack.
Brian Gerst suggested to save it in the r12 register as both r12 and rbx
are saved by the do_nmi handler as required by the C standard. But rbx
is already used for saving if swapgs needs to be run on exit of the NMI
handler.

Link: http://lkml.kernel.org/r/4FBB8C40.6080304@redhat.com
Link: http://lkml.kernel.org/r/1337763411.13348.140.camel@gandalf.stny.rr.com

Reported-by: Avi Kivity <avi@redhat.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Suggested-by: Brian Gerst <brgerst@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>

1 files changed, 20 insertions, 0 deletions

diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S
index 7d65133b51be..111f6bbd8b38 100644
--- a/arch/x86/kernel/entry_64.S
+++ b/arch/x86/kernel/entry_64.S
@@ -1758,10 +1758,30 @@ end_repeat_nmi:
          */
         call save_paranoid
         DEFAULT_FRAME 0
+
+        /*
+         * Save off the CR2 register. If we take a page fault in the NMI then
+         * it could corrupt the CR2 value. If the NMI preempts a page fault
+         * handler before it was able to read the CR2 register, and then the
+         * NMI itself takes a page fault, the page fault that was preempted
+         * will read the information from the NMI page fault and not the
+         * origin fault. Save it off and restore it if it changes.
+         * Use the r12 callee-saved register.
+         */
+        movq %cr2, %r12
+
         /* paranoidentry do_nmi, 0; without TRACE_IRQS_OFF */
         movq %rsp,%rdi
         movq $-1,%rsi
         call do_nmi
+
+        /* Did the NMI take a page fault? Restore cr2 if it did */
+        movq %cr2, %rcx
+        cmpq %rcx, %r12
+        je 1f
+        movq %r12, %cr2
+1:
+        
         testl %ebx,%ebx                         /* swapgs needed? */
         jnz nmi_restore
 nmi_swapgs: