mm: memcontrol: per-lruvec stats infrastructure

lruvecs are at the intersection of the NUMA node and memcg, which is the
scope for most paging activity.

Introduce a convenient accounting infrastructure that maintains
statistics per node, per memcg, and the lruvec itself.

Then convert over accounting sites for statistics that are already
tracked in both nodes and memcgs and can be easily switched.

[hannes@cmpxchg.org: fix crash in the new cgroup stat keeping code]
  Link: http://lkml.kernel.org/r/20170531171450.GA10481@cmpxchg.org
[hannes@cmpxchg.org: don't track uncharged pages at all
  Link: http://lkml.kernel.org/r/20170605175254.GA8547@cmpxchg.org
[hannes@cmpxchg.org: add missing free_percpu()]
  Link: http://lkml.kernel.org/r/20170605175354.GB8547@cmpxchg.org
[linux@roeck-us.net: hexagon: fix build error caused by include file order]
  Link: http://lkml.kernel.org/r/20170617153721.GA4382@roeck-us.net
Link: http://lkml.kernel.org/r/20170530181724.27197-6-hannes@cmpxchg.org
Signed-off-by: Johannes Weiner <hannes@cmpxchg.org>
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
Acked-by: Vladimir Davydov <vdavydov.dev@gmail.com>
Cc: Josef Bacik <josef@toxicpanda.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Rik van Riel <riel@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

1 files changed, 3 insertions, 6 deletions

diff --git a/mm/workingset.c b/mm/workingset.c
index b8c9ab678479..7119cd745ace 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -288,12 +288,10 @@ bool workingset_refault(void *shadow)
          */
         refault_distance = (refault - eviction) & EVICTION_MASK;
 
-        inc_node_state(pgdat, WORKINGSET_REFAULT);
-        inc_memcg_state(memcg, WORKINGSET_REFAULT);
+        inc_lruvec_state(lruvec, WORKINGSET_REFAULT);
 
         if (refault_distance <= active_file) {
-                inc_node_state(pgdat, WORKINGSET_ACTIVATE);
-                inc_memcg_state(memcg, WORKINGSET_ACTIVATE);
+                inc_lruvec_state(lruvec, WORKINGSET_ACTIVATE);
                 rcu_read_unlock();
                 return true;
         }
@@ -474,8 +472,7 @@ static enum lru_status shadow_lru_isolate(struct list_head *item,
         }
         if (WARN_ON_ONCE(node->exceptional))
                 goto out_invalid;
-        inc_node_state(page_pgdat(virt_to_page(node)), WORKINGSET_NODERECLAIM);
-        inc_memcg_page_state(virt_to_page(node), WORKINGSET_NODERECLAIM);
+        inc_lruvec_page_state(virt_to_page(node), WORKINGSET_NODERECLAIM);
         __radix_tree_delete_node(&mapping->page_tree, node,
                                  workingset_update_node, mapping);