shm: fix null pointer deref when userspace specifies invalid hugepage size

Dave reported an oops triggered by trinity:

  BUG: unable to handle kernel NULL pointer dereference at 0000000000000008
  IP: newseg+0x10d/0x390
  PGD cf8c1067 PUD cf8c2067 PMD 0
  Oops: 0000 [#1] PREEMPT SMP DEBUG_PAGEALLOC
  CPU: 2 PID: 7636 Comm: trinity-child2 Not tainted 3.9.0+#67
  ...
  Call Trace:
    ipcget+0x182/0x380
    SyS_shmget+0x5a/0x60
    tracesys+0xdd/0xe2

This bug was introduced by commit af73e4d9506d ("hugetlbfs: fix mmap
failure in unaligned size request").

Reported-by: Dave Jones <davej@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Li Zefan <lizfan@huawei.com>
Reviewed-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Acked-by: Rik van Riel <riel@redhat.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 7 insertions, 1 deletions

diff --git a/ipc/shm.c b/ipc/shm.c
index 34af1fe34701..7e199fa1960f 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -493,7 +493,13 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
         if (shmflg & SHM_HUGETLB) {
                 struct hstate *hs = hstate_sizelog((shmflg >> SHM_HUGE_SHIFT)
                                                 & SHM_HUGE_MASK);
-                size_t hugesize = ALIGN(size, huge_page_size(hs));
+                size_t hugesize;
+
+                if (!hs) {
+                        error = -EINVAL;
+                        goto no_file;
+                }
+                hugesize = ALIGN(size, huge_page_size(hs));
 
                 /* hugetlb_file_setup applies strict accounting */
                 if (shmflg & SHM_NORESERVE)