oom: avoid killing a task if a thread sharing its mm cannot be killed

The oom killer's goal is to kill a memory-hogging task so that it may
exit, free its memory, and allow the current context to allocate the
memory that triggered it in the first place.  Thus, killing a task is
pointless if other threads sharing its mm cannot be killed because of its
/proc/pid/oom_adj or /proc/pid/oom_score_adj value.

This patch checks whether any other thread sharing p->mm has an
oom_score_adj of OOM_SCORE_ADJ_MIN.  If so, the thread cannot be killed
and oom_badness(p) returns 0, meaning it's unkillable.

Signed-off-by: David Rientjes <rientjes@google.com>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Ying Han <yinghan@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 5 insertions, 4 deletions

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 4029583a1024..4395f371bc7c 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -162,10 +162,11 @@ unsigned int oom_badness(struct task_struct *p, struct mem_cgroup *mem,
                 return 0;
 
         /*
-         * Shortcut check for OOM_SCORE_ADJ_MIN so the entire heuristic doesn't
-         * need to be executed for something that cannot be killed.
+         * Shortcut check for a thread sharing p->mm that is OOM_SCORE_ADJ_MIN
+         * so the entire heuristic doesn't need to be executed for something
+         * that cannot be killed.
          */
-        if (p->signal->oom_score_adj == OOM_SCORE_ADJ_MIN) {
+        if (atomic_read(&p->mm->oom_disable_count)) {
                 task_unlock(p);
                 return 0;
         }
@@ -680,7 +681,7 @@ void out_of_memory(struct zonelist *zonelist, gfp_t gfp_mask,
         read_lock(&tasklist_lock);
         if (sysctl_oom_kill_allocating_task &&
             !oom_unkillable_task(current, NULL, nodemask) &&
-            (current->signal->oom_adj != OOM_DISABLE)) {
+            current->mm && !atomic_read(&current->mm->oom_disable_count)) {
                 /*
                  * oom_kill_process() needs tasklist_lock held.  If it returns
                  * non-zero, current could not be killed so we must fallback to