x86/entry/64: Fix irqflag tracing wrt context tracking

Paolo pointed out that enter_from_user_mode could be called
while irqflags were traced as though IRQs were on.

In principle, this could confuse lockdep.  It doesn't cause any
problems that I've seen in any configuration, but if I build
with CONFIG_DEBUG_LOCKDEP=y, enable a nohz_full CPU, and add
code like:

	if (irqs_disabled()) {
		spin_lock(&something);
		spin_unlock(&something);
	}

to the top of enter_from_user_mode, then lockdep will complain
without this fix.  It seems that lockdep's irqflags sanity
checks are too weak to detect this bug without forcing the
issue.

This patch adds one byte to normal kernels, and it's IMO a bit
ugly. I haven't spotted a better way to do this yet, though.
The issue is that we can't do TRACE_IRQS_OFF until after SWAPGS
(if needed), but we're also supposed to do it before calling C
code.

An alternative approach would be to call trace_hardirqs_off in
enter_from_user_mode.  That would be less code and would not
bloat normal kernels at all, but it would be harder to see how
the code worked.

Signed-off-by: Andy Lutomirski <luto@kernel.org>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Andy Lutomirski <luto@amacapital.net>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Brian Gerst <brgerst@gmail.com>
Cc: Denys Vlasenko <dvlasenk@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/86237e362390dfa6fec12de4d75a238acb0ae787.1447361906.git.luto@kernel.org
Signed-off-by: Ingo Molnar <mingo@kernel.org>

1 files changed, 18 insertions, 1 deletions

diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S
index 53616ca03244..a55697d19824 100644
--- a/arch/x86/entry/entry_64.S
+++ b/arch/x86/entry/entry_64.S
@@ -509,6 +509,17 @@ END(irq_entries_start)
          * tracking that we're in kernel mode.
          */
         SWAPGS
+
+        /*
+         * We need to tell lockdep that IRQs are off.  We can't do this until
+         * we fix gsbase, and we should do it before enter_from_user_mode
+         * (which can take locks).  Since TRACE_IRQS_OFF idempotent,
+         * the simplest way to handle it is to just call it twice if
+         * we enter from user mode.  There's no reason to optimize this since
+         * TRACE_IRQS_OFF is a no-op if lockdep is off.
+         */
+        TRACE_IRQS_OFF
+
 #ifdef CONFIG_CONTEXT_TRACKING
         call enter_from_user_mode
 #endif
@@ -1049,12 +1060,18 @@ ENTRY(error_entry)
         SWAPGS
 
 .Lerror_entry_from_usermode_after_swapgs:
+        /*
+         * We need to tell lockdep that IRQs are off.  We can't do this until
+         * we fix gsbase, and we should do it before enter_from_user_mode
+         * (which can take locks).
+         */
+        TRACE_IRQS_OFF
 #ifdef CONFIG_CONTEXT_TRACKING
         call enter_from_user_mode
 #endif
+        ret
 
 .Lerror_entry_done:
-
         TRACE_IRQS_OFF
         ret