memcg: enhance memcg iterator to support predicates

The caller of the iterator might know that some nodes or even subtrees
should be skipped but there is no way to tell iterators about that so the
only choice left is to let iterators to visit each node and do the
selection outside of the iterating code.  This, however, doesn't scale
well with hierarchies with many groups where only few groups are
interesting.

This patch adds mem_cgroup_iter_cond variant of the iterator with a
callback which gets called for every visited node.  There are three
possible ways how the callback can influence the walk.  Either the node is
visited, it is skipped but the tree walk continues down the tree or the
whole subtree of the current group is skipped.

[hughd@google.com: fix memcg-less page reclaim]
Signed-off-by: Michal Hocko <mhocko@suse.cz>
Cc: Balbir Singh <bsingharora@gmail.com>
Cc: Glauber Costa <glommer@openvz.org>
Cc: Greg Thelen <gthelen@google.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: Michel Lespinasse <walken@google.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Ying Han <yinghan@google.com>
Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 5 insertions, 11 deletions

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 1896e7ca494b..f2e35099508b 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2151,21 +2151,16 @@ __shrink_zone(struct zone *zone, struct scan_control *sc, bool soft_reclaim)
                         .zone = zone,
                         .priority = sc->priority,
                 };
-                struct mem_cgroup *memcg;
+                struct mem_cgroup *memcg = NULL;
+                mem_cgroup_iter_filter filter = (soft_reclaim) ?
+                        mem_cgroup_soft_reclaim_eligible : NULL;
 
                 nr_reclaimed = sc->nr_reclaimed;
                 nr_scanned = sc->nr_scanned;
 
-                memcg = mem_cgroup_iter(root, NULL, &reclaim);
-                do {
+                while ((memcg = mem_cgroup_iter_cond(root, memcg, &reclaim, filter))) {
                         struct lruvec *lruvec;
 
-                        if (soft_reclaim &&
-                            !mem_cgroup_soft_reclaim_eligible(memcg, root)) {
-                                memcg = mem_cgroup_iter(root, memcg, &reclaim);
-                                continue;
-                        }
-
                         lruvec = mem_cgroup_zone_lruvec(zone, memcg);
 
                         shrink_lruvec(lruvec, sc);
@@ -2185,8 +2180,7 @@ __shrink_zone(struct zone *zone, struct scan_control *sc, bool soft_reclaim)
                                 mem_cgroup_iter_break(root, memcg);
                                 break;
                         }
-                        memcg = mem_cgroup_iter(root, memcg, &reclaim);
-                } while (memcg);
+                }
 
                 vmpressure(sc->gfp_mask, sc->target_mem_cgroup,
                            sc->nr_scanned - nr_scanned,