shmem: get_unmapped_area align huge page

Provide a shmem_get_unmapped_area method in file_operations, called at
mmap time to decide the mapping address.  It could be conditional on
CONFIG_TRANSPARENT_HUGEPAGE, but save #ifdefs in other places by making
it unconditional.

shmem_get_unmapped_area() first calls the usual mm->get_unmapped_area
(which we treat as a black box, highly dependent on architecture and
config and executable layout).  Lots of conditions, and in most cases it
just goes with the address that chose; but when our huge stars are
rightly aligned, yet that did not provide a suitable address, go back to
ask for a larger arena, within which to align the mapping suitably.

There have to be some direct calls to shmem_get_unmapped_area(), not via
the file_operations: because of the way shmem_zero_setup() is called to
create a shmem object late in the mmap sequence, when MAP_SHARED is
requested with MAP_ANONYMOUS or /dev/zero.  Though this only matters
when /proc/sys/vm/shmem_huge has been set.

Link: http://lkml.kernel.org/r/1466021202-61880-29-git-send-email-kirill.shutemov@linux.intel.com
Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>

Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 24 insertions, 0 deletions

diff --git a/drivers/char/mem.c b/drivers/char/mem.c
index d633974e7f8b..a33163dbb913 100644
--- a/drivers/char/mem.c
+++ b/drivers/char/mem.c
@@ -22,6 +22,7 @@
 #include <linux/device.h>
 #include <linux/highmem.h>
 #include <linux/backing-dev.h>
+#include <linux/shmem_fs.h>
 #include <linux/splice.h>
 #include <linux/pfn.h>
 #include <linux/export.h>
@@ -657,6 +658,28 @@ static int mmap_zero(struct file *file, struct vm_area_struct *vma)
         return 0;
 }
 
+static unsigned long get_unmapped_area_zero(struct file *file,
+                                unsigned long addr, unsigned long len,
+                                unsigned long pgoff, unsigned long flags)
+{
+#ifdef CONFIG_MMU
+        if (flags & MAP_SHARED) {
+                /*
+                 * mmap_zero() will call shmem_zero_setup() to create a file,
+                 * so use shmem's get_unmapped_area in case it can be huge;
+                 * and pass NULL for file as in mmap.c's get_unmapped_area(),
+                 * so as not to confuse shmem with our handle on "/dev/zero".
+                 */
+                return shmem_get_unmapped_area(NULL, addr, len, pgoff, flags);
+        }
+
+        /* Otherwise flags & MAP_PRIVATE: with no shmem object beneath it */
+        return current->mm->get_unmapped_area(file, addr, len, pgoff, flags);
+#else
+        return -ENOSYS;
+#endif
+}
+
 static ssize_t write_full(struct file *file, const char __user *buf,
                           size_t count, loff_t *ppos)
 {
@@ -764,6 +787,7 @@ static const struct file_operations zero_fops = {
         .read_iter      = read_iter_zero,
         .write_iter     = write_iter_zero,
         .mmap           = mmap_zero,
+        .get_unmapped_area = get_unmapped_area_zero,
 #ifndef CONFIG_MMU
         .mmap_capabilities = zero_mmap_capabilities,
 #endif