oom: make sure that TIF_MEMDIE is set under task_lock

OOM killer tries to exclude tasks which do not have mm_struct associated
because killing such a task wouldn't help much.  The OOM victim gets
TIF_MEMDIE set to disable OOM killer while the current victim releases the
memory and then enables the OOM killer again by dropping the flag.

oom_kill_process is currently prone to a race condition when the OOM
victim is already exiting and TIF_MEMDIE is set after the task releases
its address space.  This might theoretically lead to OOM livelock if the
OOM victim blocks on an allocation later during exiting because it
wouldn't kill any other process and the exiting one won't be able to exit.
 The situation is highly unlikely because the OOM victim is expected to
release some memory which should help to sort out OOM situation.

Fix this by checking task->mm and setting TIF_MEMDIE flag under task_lock
which will serialize the OOM killer with exit_mm which sets task->mm to
NULL.  Setting the flag for current is not necessary because check and set
is not racy.

Reported-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Signed-off-by: Michal Hocko <mhocko@suse.cz>
Cc: David Rientjes <rientjes@google.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 5 insertions, 2 deletions

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index f82dd13cca68..294493a7ae4b 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -438,11 +438,14 @@ void oom_kill_process(struct task_struct *p, gfp_t gfp_mask, int order,
          * If the task is already exiting, don't alarm the sysadmin or kill
          * its children or threads, just set TIF_MEMDIE so it can die quickly
          */
-        if (task_will_free_mem(p)) {
+        task_lock(p);
+        if (p->mm && task_will_free_mem(p)) {
                 set_tsk_thread_flag(p, TIF_MEMDIE);
+                task_unlock(p);
                 put_task_struct(p);
                 return;
         }
+        task_unlock(p);
 
         if (__ratelimit(&oom_rs))
                 dump_header(p, gfp_mask, order, memcg, nodemask);
@@ -492,6 +495,7 @@ void oom_kill_process(struct task_struct *p, gfp_t gfp_mask, int order,
 
         /* mm cannot safely be dereferenced after task_unlock(victim) */
         mm = victim->mm;
+        set_tsk_thread_flag(victim, TIF_MEMDIE);
         pr_err("Killed process %d (%s) total-vm:%lukB, anon-rss:%lukB, file-rss:%lukB\n",
                 task_pid_nr(victim), victim->comm, K(victim->mm->total_vm),
                 K(get_mm_counter(victim->mm, MM_ANONPAGES)),
@@ -522,7 +526,6 @@ void oom_kill_process(struct task_struct *p, gfp_t gfp_mask, int order,
                 }
         rcu_read_unlock();
 
-        set_tsk_thread_flag(victim, TIF_MEMDIE);
         do_send_sig_info(SIGKILL, SEND_SIG_FORCED, victim, true);
         put_task_struct(victim);
 }