x86: some lock annotations for user copy paths

copy_to/from_user and all its variants (except the atomic ones) can take a
page fault and perform non-trivial work like taking mmap_sem and entering
the filesyste/pagecache.

Unfortunately, this often escapes lockdep because a common pattern is to
use it to read in some arguments just set up from userspace, or write data
back to a hot buffer. In those cases, it will be unlikely for page reclaim
to get a window in to cause copy_*_user to fault.

With the new might_lock primitives, add some annotations to x86. I don't
know if I caught all possible faulting points (it's a bit of a maze, and I
didn't really look at 32-bit). But this is a starting point.

Boots and runs OK so far.

Signed-off-by: Nick Piggin <npiggin@suse.de>
Acked-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Ingo Molnar <mingo@elte.hu>

1 files changed, 8 insertions, 2 deletions

diff --git a/include/asm-x86/uaccess_32.h b/include/asm-x86/uaccess_32.h
index 6fdef39a0bcb..d725e2d703f7 100644
--- a/include/asm-x86/uaccess_32.h
+++ b/include/asm-x86/uaccess_32.h
@@ -82,8 +82,10 @@ __copy_to_user_inatomic(void __user *to, const void *from, unsigned long n)
 static __always_inline unsigned long __must_check
 __copy_to_user(void __user *to, const void *from, unsigned long n)
 {
-       might_sleep();
-       return __copy_to_user_inatomic(to, from, n);
+        might_sleep();
+        if (current->mm)
+                might_lock_read(&current->mm->mmap_sem);
+        return __copy_to_user_inatomic(to, from, n);
 }
 
 static __always_inline unsigned long
@@ -138,6 +140,8 @@ static __always_inline unsigned long
 __copy_from_user(void *to, const void __user *from, unsigned long n)
 {
         might_sleep();
+        if (current->mm)
+                might_lock_read(&current->mm->mmap_sem);
         if (__builtin_constant_p(n)) {
                 unsigned long ret;
 
@@ -160,6 +164,8 @@ static __always_inline unsigned long __copy_from_user_nocache(void *to,
                                 const void __user *from, unsigned long n)
 {
         might_sleep();
+        if (current->mm)
+                might_lock_read(&current->mm->mmap_sem);
         if (__builtin_constant_p(n)) {
                 unsigned long ret;