memcg: avoid unnecessary system-wide-oom-killer

Current mmtom has new oom function as pagefault_out_of_memory().  It's
added for select bad process rathar than killing current.

When memcg hit limit and calls OOM at page_fault, this handler called and
system-wide-oom handling happens.  (means kernel panics if panic_on_oom is
true....)

To avoid overkill, check memcg's recent behavior before starting
system-wide-oom.

And this patch also fixes to guarantee "don't accnout against process with
TIF_MEMDIE".  This is necessary for smooth OOM.

[akpm@linux-foundation.org: build fix]
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Li Zefan <lizf@cn.fujitsu.com>
Cc: Balbir Singh <balbir@in.ibm.com>
Cc: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
Cc: Badari Pulavarty <pbadari@us.ibm.com>
Cc: Jan Blunck <jblunck@suse.de>
Cc: Hirokazu Takahashi <taka@valinux.co.jp>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 8 insertions, 0 deletions

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index 6b9e758c98a5..fd150e3a2567 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -560,6 +560,13 @@ void pagefault_out_of_memory(void)
                 /* Got some memory back in the last second. */
                 return;
 
+        /*
+         * If this is from memcg, oom-killer is already invoked.
+         * and not worth to go system-wide-oom.
+         */
+        if (mem_cgroup_oom_called(current))
+                goto rest_and_return;
+
         if (sysctl_panic_on_oom)
                 panic("out of memory from page fault. panic_on_oom is selected.\n");
 
@@ -571,6 +578,7 @@ void pagefault_out_of_memory(void)
          * Give "p" a good chance of killing itself before we
          * retry to allocate memory.
          */
+rest_and_return:
         if (!test_thread_flag(TIF_MEMDIE))
                 schedule_timeout_uninterruptible(1);
 }