memcg: fix lru rotation in isolate_pages

Try to fix memcg's lru rotation sanity: make memcg use the same logic as
the global LRU does.

Now, at __isolate_lru_page() retruns -EBUSY, the page is rotated to the
tail of LRU in global LRU's isolate LRU pages.  But in memcg, it's not
handled.  This makes memcg do the same behavior as global LRU and rotate
LRU in the page is busy.

Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Acked-by: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp>
Cc: Balbir Singh <balbir@in.ibm.com>
Cc: Mel Gorman <mel@csn.ul.ie>
Cc: Minchan Kim <minchan.kim@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 3 insertions, 1 deletions

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 4139aa52b941..e8fa2d9eb212 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -837,7 +837,6 @@ int __isolate_lru_page(struct page *page, int mode, int file)
                  */
                 ClearPageLRU(page);
                 ret = 0;
-                mem_cgroup_del_lru(page);
         }
 
         return ret;
@@ -885,12 +884,14 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan,
                 switch (__isolate_lru_page(page, mode, file)) {
                 case 0:
                         list_move(&page->lru, dst);
+                        mem_cgroup_del_lru(page);
                         nr_taken++;
                         break;
 
                 case -EBUSY:
                         /* else it is being freed elsewhere */
                         list_move(&page->lru, src);
+                        mem_cgroup_rotate_lru_list(page, page_lru(page));
                         continue;
 
                 default:
@@ -931,6 +932,7 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan,
                                 continue;
                         if (__isolate_lru_page(cursor_page, mode, file) == 0) {
                                 list_move(&cursor_page->lru, dst);
+                                mem_cgroup_del_lru(page);
                                 nr_taken++;
                                 scan++;
                         }