aboutsummaryrefslogtreecommitdiffstats
path: root/Documentation/vm
diff options
context:
space:
mode:
authorJonathan Herman <hermanjl@cs.unc.edu>2013-01-17 16:15:55 -0500
committerJonathan Herman <hermanjl@cs.unc.edu>2013-01-17 16:15:55 -0500
commit8dea78da5cee153b8af9c07a2745f6c55057fe12 (patch)
treea8f4d49d63b1ecc92f2fddceba0655b2472c5bd9 /Documentation/vm
parent406089d01562f1e2bf9f089fd7637009ebaad589 (diff)
Patched in Tegra support.
Diffstat (limited to 'Documentation/vm')
-rw-r--r--Documentation/vm/00-INDEX2
-rw-r--r--Documentation/vm/cleancache.txt43
-rw-r--r--Documentation/vm/frontswap.txt278
-rw-r--r--Documentation/vm/hugetlbpage.txt10
-rw-r--r--Documentation/vm/numa4
-rw-r--r--Documentation/vm/pagemap.txt6
-rw-r--r--Documentation/vm/slub.txt9
-rw-r--r--Documentation/vm/transhuge.txt81
-rw-r--r--Documentation/vm/unevictable-lru.txt22
9 files changed, 46 insertions, 409 deletions
diff --git a/Documentation/vm/00-INDEX b/Documentation/vm/00-INDEX
index 5481c8ba341..dca82d7c83d 100644
--- a/Documentation/vm/00-INDEX
+++ b/Documentation/vm/00-INDEX
@@ -30,6 +30,8 @@ page_migration
30 - description of page migration in NUMA systems. 30 - description of page migration in NUMA systems.
31pagemap.txt 31pagemap.txt
32 - pagemap, from the userspace perspective 32 - pagemap, from the userspace perspective
33slabinfo.c
34 - source code for a tool to get reports about slabs.
33slub.txt 35slub.txt
34 - a short users guide for SLUB. 36 - a short users guide for SLUB.
35unevictable-lru.txt 37unevictable-lru.txt
diff --git a/Documentation/vm/cleancache.txt b/Documentation/vm/cleancache.txt
index 142fbb0f325..36c367c7308 100644
--- a/Documentation/vm/cleancache.txt
+++ b/Documentation/vm/cleancache.txt
@@ -46,11 +46,10 @@ a negative return value indicates failure. A "put_page" will copy a
46the pool id, a file key, and a page index into the file. (The combination 46the pool id, a file key, and a page index into the file. (The combination
47of a pool id, a file key, and an index is sometimes called a "handle".) 47of a pool id, a file key, and an index is sometimes called a "handle".)
48A "get_page" will copy the page, if found, from cleancache into kernel memory. 48A "get_page" will copy the page, if found, from cleancache into kernel memory.
49An "invalidate_page" will ensure the page no longer is present in cleancache; 49A "flush_page" will ensure the page no longer is present in cleancache;
50an "invalidate_inode" will invalidate all pages associated with the specified 50a "flush_inode" will flush all pages associated with the specified file;
51file; and, when a filesystem is unmounted, an "invalidate_fs" will invalidate 51and, when a filesystem is unmounted, a "flush_fs" will flush all pages in
52all pages in all files specified by the given pool id and also surrender 52all files specified by the given pool id and also surrender the pool id.
53the pool id.
54 53
55An "init_shared_fs", like init_fs, obtains a pool id but tells cleancache 54An "init_shared_fs", like init_fs, obtains a pool id but tells cleancache
56to treat the pool as shared using a 128-bit UUID as a key. On systems 55to treat the pool as shared using a 128-bit UUID as a key. On systems
@@ -63,12 +62,12 @@ of the kernel (e.g. by "tools" that control cleancache). Or a
63cleancache implementation can simply disable shared_init by always 62cleancache implementation can simply disable shared_init by always
64returning a negative value. 63returning a negative value.
65 64
66If a get_page is successful on a non-shared pool, the page is invalidated 65If a get_page is successful on a non-shared pool, the page is flushed (thus
67(thus making cleancache an "exclusive" cache). On a shared pool, the page 66making cleancache an "exclusive" cache). On a shared pool, the page
68is NOT invalidated on a successful get_page so that it remains accessible to 67is NOT flushed on a successful get_page so that it remains accessible to
69other sharers. The kernel is responsible for ensuring coherency between 68other sharers. The kernel is responsible for ensuring coherency between
70cleancache (shared or not), the page cache, and the filesystem, using 69cleancache (shared or not), the page cache, and the filesystem, using
71cleancache invalidate operations as required. 70cleancache flush operations as required.
72 71
73Note that cleancache must enforce put-put-get coherency and get-get 72Note that cleancache must enforce put-put-get coherency and get-get
74coherency. For the former, if two puts are made to the same handle but 73coherency. For the former, if two puts are made to the same handle but
@@ -78,22 +77,22 @@ if a get for a given handle fails, subsequent gets for that handle will
78never succeed unless preceded by a successful put with that handle. 77never succeed unless preceded by a successful put with that handle.
79 78
80Last, cleancache provides no SMP serialization guarantees; if two 79Last, cleancache provides no SMP serialization guarantees; if two
81different Linux threads are simultaneously putting and invalidating a page 80different Linux threads are simultaneously putting and flushing a page
82with the same handle, the results are indeterminate. Callers must 81with the same handle, the results are indeterminate. Callers must
83lock the page to ensure serial behavior. 82lock the page to ensure serial behavior.
84 83
85CLEANCACHE PERFORMANCE METRICS 84CLEANCACHE PERFORMANCE METRICS
86 85
87If properly configured, monitoring of cleancache is done via debugfs in 86Cleancache monitoring is done by sysfs files in the
88the /sys/kernel/debug/mm/cleancache directory. The effectiveness of cleancache 87/sys/kernel/mm/cleancache directory. The effectiveness of cleancache
89can be measured (across all filesystems) with: 88can be measured (across all filesystems) with:
90 89
91succ_gets - number of gets that were successful 90succ_gets - number of gets that were successful
92failed_gets - number of gets that failed 91failed_gets - number of gets that failed
93puts - number of puts attempted (all "succeed") 92puts - number of puts attempted (all "succeed")
94invalidates - number of invalidates attempted 93flushes - number of flushes attempted
95 94
96A backend implementation may provide additional metrics. 95A backend implementatation may provide additional metrics.
97 96
98FAQ 97FAQ
99 98
@@ -144,7 +143,7 @@ systems.
144 143
145The core hooks for cleancache in VFS are in most cases a single line 144The core hooks for cleancache in VFS are in most cases a single line
146and the minimum set are placed precisely where needed to maintain 145and the minimum set are placed precisely where needed to maintain
147coherency (via cleancache_invalidate operations) between cleancache, 146coherency (via cleancache_flush operations) between cleancache,
148the page cache, and disk. All hooks compile into nothingness if 147the page cache, and disk. All hooks compile into nothingness if
149cleancache is config'ed off and turn into a function-pointer- 148cleancache is config'ed off and turn into a function-pointer-
150compare-to-NULL if config'ed on but no backend claims the ops 149compare-to-NULL if config'ed on but no backend claims the ops
@@ -185,15 +184,15 @@ or for real kernel-addressable RAM, it makes perfect sense for
185transcendent memory. 184transcendent memory.
186 185
1874) Why is non-shared cleancache "exclusive"? And where is the 1864) Why is non-shared cleancache "exclusive"? And where is the
188 page "invalidated" after a "get"? (Minchan Kim) 187 page "flushed" after a "get"? (Minchan Kim)
189 188
190The main reason is to free up space in transcendent memory and 189The main reason is to free up space in transcendent memory and
191to avoid unnecessary cleancache_invalidate calls. If you want inclusive, 190to avoid unnecessary cleancache_flush calls. If you want inclusive,
192the page can be "put" immediately following the "get". If 191the page can be "put" immediately following the "get". If
193put-after-get for inclusive becomes common, the interface could 192put-after-get for inclusive becomes common, the interface could
194be easily extended to add a "get_no_invalidate" call. 193be easily extended to add a "get_no_flush" call.
195 194
196The invalidate is done by the cleancache backend implementation. 195The flush is done by the cleancache backend implementation.
197 196
1985) What's the performance impact? 1975) What's the performance impact?
199 198
@@ -223,7 +222,7 @@ Some points for a filesystem to consider:
223 as tmpfs should not enable cleancache) 222 as tmpfs should not enable cleancache)
224- To ensure coherency/correctness, the FS must ensure that all 223- To ensure coherency/correctness, the FS must ensure that all
225 file removal or truncation operations either go through VFS or 224 file removal or truncation operations either go through VFS or
226 add hooks to do the equivalent cleancache "invalidate" operations 225 add hooks to do the equivalent cleancache "flush" operations
227- To ensure coherency/correctness, either inode numbers must 226- To ensure coherency/correctness, either inode numbers must
228 be unique across the lifetime of the on-disk file OR the 227 be unique across the lifetime of the on-disk file OR the
229 FS must provide an "encode_fh" function. 228 FS must provide an "encode_fh" function.
@@ -244,11 +243,11 @@ If cleancache would use the inode virtual address instead of
244inode/filehandle, the pool id could be eliminated. But, this 243inode/filehandle, the pool id could be eliminated. But, this
245won't work because cleancache retains pagecache data pages 244won't work because cleancache retains pagecache data pages
246persistently even when the inode has been pruned from the 245persistently even when the inode has been pruned from the
247inode unused list, and only invalidates the data page if the file 246inode unused list, and only flushes the data page if the file
248gets removed/truncated. So if cleancache used the inode kva, 247gets removed/truncated. So if cleancache used the inode kva,
249there would be potential coherency issues if/when the inode 248there would be potential coherency issues if/when the inode
250kva is reused for a different file. Alternately, if cleancache 249kva is reused for a different file. Alternately, if cleancache
251invalidated the pages when the inode kva was freed, much of the value 250flushed the pages when the inode kva was freed, much of the value
252of cleancache would be lost because the cache of pages in cleanache 251of cleancache would be lost because the cache of pages in cleanache
253is potentially much larger than the kernel pagecache and is most 252is potentially much larger than the kernel pagecache and is most
254useful if the pages survive inode cache removal. 253useful if the pages survive inode cache removal.
diff --git a/Documentation/vm/frontswap.txt b/Documentation/vm/frontswap.txt
deleted file mode 100644
index c71a019be60..00000000000
--- a/Documentation/vm/frontswap.txt
+++ /dev/null
@@ -1,278 +0,0 @@
1Frontswap provides a "transcendent memory" interface for swap pages.
2In some environments, dramatic performance savings may be obtained because
3swapped pages are saved in RAM (or a RAM-like device) instead of a swap disk.
4
5(Note, frontswap -- and cleancache (merged at 3.0) -- are the "frontends"
6and the only necessary changes to the core kernel for transcendent memory;
7all other supporting code -- the "backends" -- is implemented as drivers.
8See the LWN.net article "Transcendent memory in a nutshell" for a detailed
9overview of frontswap and related kernel parts:
10https://lwn.net/Articles/454795/ )
11
12Frontswap is so named because it can be thought of as the opposite of
13a "backing" store for a swap device. The storage is assumed to be
14a synchronous concurrency-safe page-oriented "pseudo-RAM device" conforming
15to the requirements of transcendent memory (such as Xen's "tmem", or
16in-kernel compressed memory, aka "zcache", or future RAM-like devices);
17this pseudo-RAM device is not directly accessible or addressable by the
18kernel and is of unknown and possibly time-varying size. The driver
19links itself to frontswap by calling frontswap_register_ops to set the
20frontswap_ops funcs appropriately and the functions it provides must
21conform to certain policies as follows:
22
23An "init" prepares the device to receive frontswap pages associated
24with the specified swap device number (aka "type"). A "store" will
25copy the page to transcendent memory and associate it with the type and
26offset associated with the page. A "load" will copy the page, if found,
27from transcendent memory into kernel memory, but will NOT remove the page
28from transcendent memory. An "invalidate_page" will remove the page
29from transcendent memory and an "invalidate_area" will remove ALL pages
30associated with the swap type (e.g., like swapoff) and notify the "device"
31to refuse further stores with that swap type.
32
33Once a page is successfully stored, a matching load on the page will normally
34succeed. So when the kernel finds itself in a situation where it needs
35to swap out a page, it first attempts to use frontswap. If the store returns
36success, the data has been successfully saved to transcendent memory and
37a disk write and, if the data is later read back, a disk read are avoided.
38If a store returns failure, transcendent memory has rejected the data, and the
39page can be written to swap as usual.
40
41If a backend chooses, frontswap can be configured as a "writethrough
42cache" by calling frontswap_writethrough(). In this mode, the reduction
43in swap device writes is lost (and also a non-trivial performance advantage)
44in order to allow the backend to arbitrarily "reclaim" space used to
45store frontswap pages to more completely manage its memory usage.
46
47Note that if a page is stored and the page already exists in transcendent memory
48(a "duplicate" store), either the store succeeds and the data is overwritten,
49or the store fails AND the page is invalidated. This ensures stale data may
50never be obtained from frontswap.
51
52If properly configured, monitoring of frontswap is done via debugfs in
53the /sys/kernel/debug/frontswap directory. The effectiveness of
54frontswap can be measured (across all swap devices) with:
55
56failed_stores - how many store attempts have failed
57loads - how many loads were attempted (all should succeed)
58succ_stores - how many store attempts have succeeded
59invalidates - how many invalidates were attempted
60
61A backend implementation may provide additional metrics.
62
63FAQ
64
651) Where's the value?
66
67When a workload starts swapping, performance falls through the floor.
68Frontswap significantly increases performance in many such workloads by
69providing a clean, dynamic interface to read and write swap pages to
70"transcendent memory" that is otherwise not directly addressable to the kernel.
71This interface is ideal when data is transformed to a different form
72and size (such as with compression) or secretly moved (as might be
73useful for write-balancing for some RAM-like devices). Swap pages (and
74evicted page-cache pages) are a great use for this kind of slower-than-RAM-
75but-much-faster-than-disk "pseudo-RAM device" and the frontswap (and
76cleancache) interface to transcendent memory provides a nice way to read
77and write -- and indirectly "name" -- the pages.
78
79Frontswap -- and cleancache -- with a fairly small impact on the kernel,
80provides a huge amount of flexibility for more dynamic, flexible RAM
81utilization in various system configurations:
82
83In the single kernel case, aka "zcache", pages are compressed and
84stored in local memory, thus increasing the total anonymous pages
85that can be safely kept in RAM. Zcache essentially trades off CPU
86cycles used in compression/decompression for better memory utilization.
87Benchmarks have shown little or no impact when memory pressure is
88low while providing a significant performance improvement (25%+)
89on some workloads under high memory pressure.
90
91"RAMster" builds on zcache by adding "peer-to-peer" transcendent memory
92support for clustered systems. Frontswap pages are locally compressed
93as in zcache, but then "remotified" to another system's RAM. This
94allows RAM to be dynamically load-balanced back-and-forth as needed,
95i.e. when system A is overcommitted, it can swap to system B, and
96vice versa. RAMster can also be configured as a memory server so
97many servers in a cluster can swap, dynamically as needed, to a single
98server configured with a large amount of RAM... without pre-configuring
99how much of the RAM is available for each of the clients!
100
101In the virtual case, the whole point of virtualization is to statistically
102multiplex physical resources across the varying demands of multiple
103virtual machines. This is really hard to do with RAM and efforts to do
104it well with no kernel changes have essentially failed (except in some
105well-publicized special-case workloads).
106Specifically, the Xen Transcendent Memory backend allows otherwise
107"fallow" hypervisor-owned RAM to not only be "time-shared" between multiple
108virtual machines, but the pages can be compressed and deduplicated to
109optimize RAM utilization. And when guest OS's are induced to surrender
110underutilized RAM (e.g. with "selfballooning"), sudden unexpected
111memory pressure may result in swapping; frontswap allows those pages
112to be swapped to and from hypervisor RAM (if overall host system memory
113conditions allow), thus mitigating the potentially awful performance impact
114of unplanned swapping.
115
116A KVM implementation is underway and has been RFC'ed to lkml. And,
117using frontswap, investigation is also underway on the use of NVM as
118a memory extension technology.
119
1202) Sure there may be performance advantages in some situations, but
121 what's the space/time overhead of frontswap?
122
123If CONFIG_FRONTSWAP is disabled, every frontswap hook compiles into
124nothingness and the only overhead is a few extra bytes per swapon'ed
125swap device. If CONFIG_FRONTSWAP is enabled but no frontswap "backend"
126registers, there is one extra global variable compared to zero for
127every swap page read or written. If CONFIG_FRONTSWAP is enabled
128AND a frontswap backend registers AND the backend fails every "store"
129request (i.e. provides no memory despite claiming it might),
130CPU overhead is still negligible -- and since every frontswap fail
131precedes a swap page write-to-disk, the system is highly likely
132to be I/O bound and using a small fraction of a percent of a CPU
133will be irrelevant anyway.
134
135As for space, if CONFIG_FRONTSWAP is enabled AND a frontswap backend
136registers, one bit is allocated for every swap page for every swap
137device that is swapon'd. This is added to the EIGHT bits (which
138was sixteen until about 2.6.34) that the kernel already allocates
139for every swap page for every swap device that is swapon'd. (Hugh
140Dickins has observed that frontswap could probably steal one of
141the existing eight bits, but let's worry about that minor optimization
142later.) For very large swap disks (which are rare) on a standard
1434K pagesize, this is 1MB per 32GB swap.
144
145When swap pages are stored in transcendent memory instead of written
146out to disk, there is a side effect that this may create more memory
147pressure that can potentially outweigh the other advantages. A
148backend, such as zcache, must implement policies to carefully (but
149dynamically) manage memory limits to ensure this doesn't happen.
150
1513) OK, how about a quick overview of what this frontswap patch does
152 in terms that a kernel hacker can grok?
153
154Let's assume that a frontswap "backend" has registered during
155kernel initialization; this registration indicates that this
156frontswap backend has access to some "memory" that is not directly
157accessible by the kernel. Exactly how much memory it provides is
158entirely dynamic and random.
159
160Whenever a swap-device is swapon'd frontswap_init() is called,
161passing the swap device number (aka "type") as a parameter.
162This notifies frontswap to expect attempts to "store" swap pages
163associated with that number.
164
165Whenever the swap subsystem is readying a page to write to a swap
166device (c.f swap_writepage()), frontswap_store is called. Frontswap
167consults with the frontswap backend and if the backend says it does NOT
168have room, frontswap_store returns -1 and the kernel swaps the page
169to the swap device as normal. Note that the response from the frontswap
170backend is unpredictable to the kernel; it may choose to never accept a
171page, it could accept every ninth page, or it might accept every
172page. But if the backend does accept a page, the data from the page
173has already been copied and associated with the type and offset,
174and the backend guarantees the persistence of the data. In this case,
175frontswap sets a bit in the "frontswap_map" for the swap device
176corresponding to the page offset on the swap device to which it would
177otherwise have written the data.
178
179When the swap subsystem needs to swap-in a page (swap_readpage()),
180it first calls frontswap_load() which checks the frontswap_map to
181see if the page was earlier accepted by the frontswap backend. If
182it was, the page of data is filled from the frontswap backend and
183the swap-in is complete. If not, the normal swap-in code is
184executed to obtain the page of data from the real swap device.
185
186So every time the frontswap backend accepts a page, a swap device read
187and (potentially) a swap device write are replaced by a "frontswap backend
188store" and (possibly) a "frontswap backend loads", which are presumably much
189faster.
190
1914) Can't frontswap be configured as a "special" swap device that is
192 just higher priority than any real swap device (e.g. like zswap,
193 or maybe swap-over-nbd/NFS)?
194
195No. First, the existing swap subsystem doesn't allow for any kind of
196swap hierarchy. Perhaps it could be rewritten to accommodate a hierarchy,
197but this would require fairly drastic changes. Even if it were
198rewritten, the existing swap subsystem uses the block I/O layer which
199assumes a swap device is fixed size and any page in it is linearly
200addressable. Frontswap barely touches the existing swap subsystem,
201and works around the constraints of the block I/O subsystem to provide
202a great deal of flexibility and dynamicity.
203
204For example, the acceptance of any swap page by the frontswap backend is
205entirely unpredictable. This is critical to the definition of frontswap
206backends because it grants completely dynamic discretion to the
207backend. In zcache, one cannot know a priori how compressible a page is.
208"Poorly" compressible pages can be rejected, and "poorly" can itself be
209defined dynamically depending on current memory constraints.
210
211Further, frontswap is entirely synchronous whereas a real swap
212device is, by definition, asynchronous and uses block I/O. The
213block I/O layer is not only unnecessary, but may perform "optimizations"
214that are inappropriate for a RAM-oriented device including delaying
215the write of some pages for a significant amount of time. Synchrony is
216required to ensure the dynamicity of the backend and to avoid thorny race
217conditions that would unnecessarily and greatly complicate frontswap
218and/or the block I/O subsystem. That said, only the initial "store"
219and "load" operations need be synchronous. A separate asynchronous thread
220is free to manipulate the pages stored by frontswap. For example,
221the "remotification" thread in RAMster uses standard asynchronous
222kernel sockets to move compressed frontswap pages to a remote machine.
223Similarly, a KVM guest-side implementation could do in-guest compression
224and use "batched" hypercalls.
225
226In a virtualized environment, the dynamicity allows the hypervisor
227(or host OS) to do "intelligent overcommit". For example, it can
228choose to accept pages only until host-swapping might be imminent,
229then force guests to do their own swapping.
230
231There is a downside to the transcendent memory specifications for
232frontswap: Since any "store" might fail, there must always be a real
233slot on a real swap device to swap the page. Thus frontswap must be
234implemented as a "shadow" to every swapon'd device with the potential
235capability of holding every page that the swap device might have held
236and the possibility that it might hold no pages at all. This means
237that frontswap cannot contain more pages than the total of swapon'd
238swap devices. For example, if NO swap device is configured on some
239installation, frontswap is useless. Swapless portable devices
240can still use frontswap but a backend for such devices must configure
241some kind of "ghost" swap device and ensure that it is never used.
242
2435) Why this weird definition about "duplicate stores"? If a page
244 has been previously successfully stored, can't it always be
245 successfully overwritten?
246
247Nearly always it can, but no, sometimes it cannot. Consider an example
248where data is compressed and the original 4K page has been compressed
249to 1K. Now an attempt is made to overwrite the page with data that
250is non-compressible and so would take the entire 4K. But the backend
251has no more space. In this case, the store must be rejected. Whenever
252frontswap rejects a store that would overwrite, it also must invalidate
253the old data and ensure that it is no longer accessible. Since the
254swap subsystem then writes the new data to the read swap device,
255this is the correct course of action to ensure coherency.
256
2576) What is frontswap_shrink for?
258
259When the (non-frontswap) swap subsystem swaps out a page to a real
260swap device, that page is only taking up low-value pre-allocated disk
261space. But if frontswap has placed a page in transcendent memory, that
262page may be taking up valuable real estate. The frontswap_shrink
263routine allows code outside of the swap subsystem to force pages out
264of the memory managed by frontswap and back into kernel-addressable memory.
265For example, in RAMster, a "suction driver" thread will attempt
266to "repatriate" pages sent to a remote machine back to the local machine;
267this is driven using the frontswap_shrink mechanism when memory pressure
268subsides.
269
2707) Why does the frontswap patch create the new include file swapfile.h?
271
272The frontswap code depends on some swap-subsystem-internal data
273structures that have, over the years, moved back and forth between
274static and global. This seemed a reasonable compromise: Define
275them as global but declare them in a new include file that isn't
276included by the large number of source files that include swap.h.
277
278Dan Magenheimer, last updated April 9, 2012
diff --git a/Documentation/vm/hugetlbpage.txt b/Documentation/vm/hugetlbpage.txt
index 4ac359b7aa1..f8551b3879f 100644
--- a/Documentation/vm/hugetlbpage.txt
+++ b/Documentation/vm/hugetlbpage.txt
@@ -299,17 +299,11 @@ map_hugetlb.c.
299******************************************************************* 299*******************************************************************
300 300
301/* 301/*
302 * map_hugetlb: see tools/testing/selftests/vm/map_hugetlb.c 302 * hugepage-shm: see Documentation/vm/hugepage-shm.c
303 */ 303 */
304 304
305******************************************************************* 305*******************************************************************
306 306
307/* 307/*
308 * hugepage-shm: see tools/testing/selftests/vm/hugepage-shm.c 308 * hugepage-mmap: see Documentation/vm/hugepage-mmap.c
309 */
310
311*******************************************************************
312
313/*
314 * hugepage-mmap: see tools/testing/selftests/vm/hugepage-mmap.c
315 */ 309 */
diff --git a/Documentation/vm/numa b/Documentation/vm/numa
index ade01274212..a200a386429 100644
--- a/Documentation/vm/numa
+++ b/Documentation/vm/numa
@@ -109,11 +109,11 @@ to improve NUMA locality using various CPU affinity command line interfaces,
109such as taskset(1) and numactl(1), and program interfaces such as 109such as taskset(1) and numactl(1), and program interfaces such as
110sched_setaffinity(2). Further, one can modify the kernel's default local 110sched_setaffinity(2). Further, one can modify the kernel's default local
111allocation behavior using Linux NUMA memory policy. 111allocation behavior using Linux NUMA memory policy.
112[see Documentation/vm/numa_memory_policy.txt.] 112[see Documentation/vm/numa_memory_policy.]
113 113
114System administrators can restrict the CPUs and nodes' memories that a non- 114System administrators can restrict the CPUs and nodes' memories that a non-
115privileged user can specify in the scheduling or NUMA commands and functions 115privileged user can specify in the scheduling or NUMA commands and functions
116using control groups and CPUsets. [see Documentation/cgroups/cpusets.txt] 116using control groups and CPUsets. [see Documentation/cgroups/CPUsets.txt]
117 117
118On architectures that do not hide memoryless nodes, Linux will include only 118On architectures that do not hide memoryless nodes, Linux will include only
119zones [nodes] with memory in the zonelists. This means that for a memoryless 119zones [nodes] with memory in the zonelists. This means that for a memoryless
diff --git a/Documentation/vm/pagemap.txt b/Documentation/vm/pagemap.txt
index 7587493c67f..df09b9650a8 100644
--- a/Documentation/vm/pagemap.txt
+++ b/Documentation/vm/pagemap.txt
@@ -16,7 +16,7 @@ There are three components to pagemap:
16 * Bits 0-4 swap type if swapped 16 * Bits 0-4 swap type if swapped
17 * Bits 5-54 swap offset if swapped 17 * Bits 5-54 swap offset if swapped
18 * Bits 55-60 page shift (page size = 1<<page shift) 18 * Bits 55-60 page shift (page size = 1<<page shift)
19 * Bit 61 page is file-page or shared-anon 19 * Bit 61 reserved for future use
20 * Bit 62 page swapped 20 * Bit 62 page swapped
21 * Bit 63 page present 21 * Bit 63 page present
22 22
@@ -60,7 +60,6 @@ There are three components to pagemap:
60 19. HWPOISON 60 19. HWPOISON
61 20. NOPAGE 61 20. NOPAGE
62 21. KSM 62 21. KSM
63 22. THP
64 63
65Short descriptions to the page flags: 64Short descriptions to the page flags:
66 65
@@ -98,9 +97,6 @@ Short descriptions to the page flags:
9821. KSM 9721. KSM
99 identical memory pages dynamically shared between one or more processes 98 identical memory pages dynamically shared between one or more processes
100 99
10122. THP
102 contiguous pages which construct transparent hugepages
103
104 [IO related page flags] 100 [IO related page flags]
105 1. ERROR IO error occurred 101 1. ERROR IO error occurred
106 3. UPTODATE page has up-to-date data 102 3. UPTODATE page has up-to-date data
diff --git a/Documentation/vm/slub.txt b/Documentation/vm/slub.txt
index b0c6d1bbb43..07375e73981 100644
--- a/Documentation/vm/slub.txt
+++ b/Documentation/vm/slub.txt
@@ -17,7 +17,7 @@ data and perform operation on the slabs. By default slabinfo only lists
17slabs that have data in them. See "slabinfo -h" for more options when 17slabs that have data in them. See "slabinfo -h" for more options when
18running the command. slabinfo can be compiled with 18running the command. slabinfo can be compiled with
19 19
20gcc -o slabinfo tools/vm/slabinfo.c 20gcc -o slabinfo Documentation/vm/slabinfo.c
21 21
22Some of the modes of operation of slabinfo require that slub debugging 22Some of the modes of operation of slabinfo require that slub debugging
23be enabled on the command line. F.e. no tracking information will be 23be enabled on the command line. F.e. no tracking information will be
@@ -117,7 +117,7 @@ can be influenced by kernel parameters:
117 117
118slub_min_objects=x (default 4) 118slub_min_objects=x (default 4)
119slub_min_order=x (default 0) 119slub_min_order=x (default 0)
120slub_max_order=x (default 3 (PAGE_ALLOC_COSTLY_ORDER)) 120slub_max_order=x (default 1)
121 121
122slub_min_objects allows to specify how many objects must at least fit 122slub_min_objects allows to specify how many objects must at least fit
123into one slab in order for the allocation order to be acceptable. 123into one slab in order for the allocation order to be acceptable.
@@ -131,10 +131,7 @@ slub_min_objects.
131slub_max_order specified the order at which slub_min_objects should no 131slub_max_order specified the order at which slub_min_objects should no
132longer be checked. This is useful to avoid SLUB trying to generate 132longer be checked. This is useful to avoid SLUB trying to generate
133super large order pages to fit slub_min_objects of a slab cache with 133super large order pages to fit slub_min_objects of a slab cache with
134large object sizes into one high order page. Setting command line 134large object sizes into one high order page.
135parameter debug_guardpage_minorder=N (N > 0), forces setting
136slub_max_order to 0, what cause minimum possible order of slabs
137allocation.
138 135
139SLUB Debug output 136SLUB Debug output
140----------------- 137-----------------
diff --git a/Documentation/vm/transhuge.txt b/Documentation/vm/transhuge.txt
index 8785fb87d9c..29bdf62aac0 100644
--- a/Documentation/vm/transhuge.txt
+++ b/Documentation/vm/transhuge.txt
@@ -116,13 +116,6 @@ echo always >/sys/kernel/mm/transparent_hugepage/defrag
116echo madvise >/sys/kernel/mm/transparent_hugepage/defrag 116echo madvise >/sys/kernel/mm/transparent_hugepage/defrag
117echo never >/sys/kernel/mm/transparent_hugepage/defrag 117echo never >/sys/kernel/mm/transparent_hugepage/defrag
118 118
119By default kernel tries to use huge zero page on read page fault.
120It's possible to disable huge zero page by writing 0 or enable it
121back by writing 1:
122
123echo 0 >/sys/kernel/mm/transparent_hugepage/khugepaged/use_zero_page
124echo 1 >/sys/kernel/mm/transparent_hugepage/khugepaged/use_zero_page
125
126khugepaged will be automatically started when 119khugepaged will be automatically started when
127transparent_hugepage/enabled is set to "always" or "madvise, and it'll 120transparent_hugepage/enabled is set to "always" or "madvise, and it'll
128be automatically shutdown if it's set to "never". 121be automatically shutdown if it's set to "never".
@@ -173,76 +166,6 @@ behavior. So to make them effective you need to restart any
173application that could have been using hugepages. This also applies to 166application that could have been using hugepages. This also applies to
174the regions registered in khugepaged. 167the regions registered in khugepaged.
175 168
176== Monitoring usage ==
177
178The number of transparent huge pages currently used by the system is
179available by reading the AnonHugePages field in /proc/meminfo. To
180identify what applications are using transparent huge pages, it is
181necessary to read /proc/PID/smaps and count the AnonHugePages fields
182for each mapping. Note that reading the smaps file is expensive and
183reading it frequently will incur overhead.
184
185There are a number of counters in /proc/vmstat that may be used to
186monitor how successfully the system is providing huge pages for use.
187
188thp_fault_alloc is incremented every time a huge page is successfully
189 allocated to handle a page fault. This applies to both the
190 first time a page is faulted and for COW faults.
191
192thp_collapse_alloc is incremented by khugepaged when it has found
193 a range of pages to collapse into one huge page and has
194 successfully allocated a new huge page to store the data.
195
196thp_fault_fallback is incremented if a page fault fails to allocate
197 a huge page and instead falls back to using small pages.
198
199thp_collapse_alloc_failed is incremented if khugepaged found a range
200 of pages that should be collapsed into one huge page but failed
201 the allocation.
202
203thp_split is incremented every time a huge page is split into base
204 pages. This can happen for a variety of reasons but a common
205 reason is that a huge page is old and is being reclaimed.
206
207thp_zero_page_alloc is incremented every time a huge zero page is
208 successfully allocated. It includes allocations which where
209 dropped due race with other allocation. Note, it doesn't count
210 every map of the huge zero page, only its allocation.
211
212thp_zero_page_alloc_failed is incremented if kernel fails to allocate
213 huge zero page and falls back to using small pages.
214
215As the system ages, allocating huge pages may be expensive as the
216system uses memory compaction to copy data around memory to free a
217huge page for use. There are some counters in /proc/vmstat to help
218monitor this overhead.
219
220compact_stall is incremented every time a process stalls to run
221 memory compaction so that a huge page is free for use.
222
223compact_success is incremented if the system compacted memory and
224 freed a huge page for use.
225
226compact_fail is incremented if the system tries to compact memory
227 but failed.
228
229compact_pages_moved is incremented each time a page is moved. If
230 this value is increasing rapidly, it implies that the system
231 is copying a lot of data to satisfy the huge page allocation.
232 It is possible that the cost of copying exceeds any savings
233 from reduced TLB misses.
234
235compact_pagemigrate_failed is incremented when the underlying mechanism
236 for moving a page failed.
237
238compact_blocks_moved is incremented each time memory compaction examines
239 a huge page aligned range of pages.
240
241It is possible to establish how long the stalls were using the function
242tracer to record how long was spent in __alloc_pages_nodemask and
243using the mm_page_alloc tracepoint to identify which allocations were
244for huge pages.
245
246== get_user_pages and follow_page == 169== get_user_pages and follow_page ==
247 170
248get_user_pages and follow_page if run on a hugepage, will return the 171get_user_pages and follow_page if run on a hugepage, will return the
@@ -291,7 +214,7 @@ unaffected. libhugetlbfs will also work fine as usual.
291== Graceful fallback == 214== Graceful fallback ==
292 215
293Code walking pagetables but unware about huge pmds can simply call 216Code walking pagetables but unware about huge pmds can simply call
294split_huge_page_pmd(vma, addr, pmd) where the pmd is the one returned by 217split_huge_page_pmd(mm, pmd) where the pmd is the one returned by
295pmd_offset. It's trivial to make the code transparent hugepage aware 218pmd_offset. It's trivial to make the code transparent hugepage aware
296by just grepping for "pmd_offset" and adding split_huge_page_pmd where 219by just grepping for "pmd_offset" and adding split_huge_page_pmd where
297missing after pmd_offset returns the pmd. Thanks to the graceful 220missing after pmd_offset returns the pmd. Thanks to the graceful
@@ -314,7 +237,7 @@ diff --git a/mm/mremap.c b/mm/mremap.c
314 return NULL; 237 return NULL;
315 238
316 pmd = pmd_offset(pud, addr); 239 pmd = pmd_offset(pud, addr);
317+ split_huge_page_pmd(vma, addr, pmd); 240+ split_huge_page_pmd(mm, pmd);
318 if (pmd_none_or_clear_bad(pmd)) 241 if (pmd_none_or_clear_bad(pmd))
319 return NULL; 242 return NULL;
320 243
diff --git a/Documentation/vm/unevictable-lru.txt b/Documentation/vm/unevictable-lru.txt
index a68db7692ee..97bae3c576c 100644
--- a/Documentation/vm/unevictable-lru.txt
+++ b/Documentation/vm/unevictable-lru.txt
@@ -197,8 +197,12 @@ the pages are also "rescued" from the unevictable list in the process of
197freeing them. 197freeing them.
198 198
199page_evictable() also checks for mlocked pages by testing an additional page 199page_evictable() also checks for mlocked pages by testing an additional page
200flag, PG_mlocked (as wrapped by PageMlocked()), which is set when a page is 200flag, PG_mlocked (as wrapped by PageMlocked()). If the page is NOT mlocked,
201faulted into a VM_LOCKED vma, or found in a vma being VM_LOCKED. 201and a non-NULL VMA is supplied, page_evictable() will check whether the VMA is
202VM_LOCKED via is_mlocked_vma(). is_mlocked_vma() will SetPageMlocked() and
203update the appropriate statistics if the vma is VM_LOCKED. This method allows
204efficient "culling" of pages in the fault path that are being faulted in to
205VM_LOCKED VMAs.
202 206
203 207
204VMSCAN'S HANDLING OF UNEVICTABLE PAGES 208VMSCAN'S HANDLING OF UNEVICTABLE PAGES
@@ -367,8 +371,8 @@ mlock_fixup() filters several classes of "special" VMAs:
367 mlock_fixup() will call make_pages_present() in the hugetlbfs VMA range to 371 mlock_fixup() will call make_pages_present() in the hugetlbfs VMA range to
368 allocate the huge pages and populate the ptes. 372 allocate the huge pages and populate the ptes.
369 373
3703) VMAs with VM_DONTEXPAND are generally userspace mappings of kernel pages, 3743) VMAs with VM_DONTEXPAND or VM_RESERVED are generally userspace mappings of
371 such as the VDSO page, relay channel pages, etc. These pages 375 kernel pages, such as the VDSO page, relay channel pages, etc. These pages
372 are inherently unevictable and are not managed on the LRU lists. 376 are inherently unevictable and are not managed on the LRU lists.
373 mlock_fixup() treats these VMAs the same as hugetlbfs VMAs. It calls 377 mlock_fixup() treats these VMAs the same as hugetlbfs VMAs. It calls
374 make_pages_present() to populate the ptes. 378 make_pages_present() to populate the ptes.
@@ -534,7 +538,7 @@ different reverse map mechanisms.
534 process because mlocked pages are migratable. However, for reclaim, if 538 process because mlocked pages are migratable. However, for reclaim, if
535 the page is mapped into a VM_LOCKED VMA, the scan stops. 539 the page is mapped into a VM_LOCKED VMA, the scan stops.
536 540
537 try_to_unmap_anon() attempts to acquire in read mode the mmap semaphore of 541 try_to_unmap_anon() attempts to acquire in read mode the mmap semphore of
538 the mm_struct to which the VMA belongs. If this is successful, it will 542 the mm_struct to which the VMA belongs. If this is successful, it will
539 mlock the page via mlock_vma_page() - we wouldn't have gotten to 543 mlock the page via mlock_vma_page() - we wouldn't have gotten to
540 try_to_unmap_anon() if the page were already mlocked - and will return 544 try_to_unmap_anon() if the page were already mlocked - and will return
@@ -615,11 +619,11 @@ all PTEs from the page. For this purpose, the unevictable/mlock infrastructure
615introduced a variant of try_to_unmap() called try_to_munlock(). 619introduced a variant of try_to_unmap() called try_to_munlock().
616 620
617try_to_munlock() calls the same functions as try_to_unmap() for anonymous and 621try_to_munlock() calls the same functions as try_to_unmap() for anonymous and
618mapped file pages with an additional argument specifying unlock versus unmap 622mapped file pages with an additional argument specifing unlock versus unmap
619processing. Again, these functions walk the respective reverse maps looking 623processing. Again, these functions walk the respective reverse maps looking
620for VM_LOCKED VMAs. When such a VMA is found for anonymous pages and file 624for VM_LOCKED VMAs. When such a VMA is found for anonymous pages and file
621pages mapped in linear VMAs, as in the try_to_unmap() case, the functions 625pages mapped in linear VMAs, as in the try_to_unmap() case, the functions
622attempt to acquire the associated mmap semaphore, mlock the page via 626attempt to acquire the associated mmap semphore, mlock the page via
623mlock_vma_page() and return SWAP_MLOCK. This effectively undoes the 627mlock_vma_page() and return SWAP_MLOCK. This effectively undoes the
624pre-clearing of the page's PG_mlocked done by munlock_vma_page. 628pre-clearing of the page's PG_mlocked done by munlock_vma_page.
625 629
@@ -637,7 +641,7 @@ with it - the usual fallback position.
637Note that try_to_munlock()'s reverse map walk must visit every VMA in a page's 641Note that try_to_munlock()'s reverse map walk must visit every VMA in a page's
638reverse map to determine that a page is NOT mapped into any VM_LOCKED VMA. 642reverse map to determine that a page is NOT mapped into any VM_LOCKED VMA.
639However, the scan can terminate when it encounters a VM_LOCKED VMA and can 643However, the scan can terminate when it encounters a VM_LOCKED VMA and can
640successfully acquire the VMA's mmap semaphore for read and mlock the page. 644successfully acquire the VMA's mmap semphore for read and mlock the page.
641Although try_to_munlock() might be called a great many times when munlocking a 645Although try_to_munlock() might be called a great many times when munlocking a
642large region or tearing down a large address space that has been mlocked via 646large region or tearing down a large address space that has been mlocked via
643mlockall(), overall this is a fairly rare event. 647mlockall(), overall this is a fairly rare event.
@@ -647,7 +651,7 @@ PAGE RECLAIM IN shrink_*_list()
647------------------------------- 651-------------------------------
648 652
649shrink_active_list() culls any obviously unevictable pages - i.e. 653shrink_active_list() culls any obviously unevictable pages - i.e.
650!page_evictable(page) - diverting these to the unevictable list. 654!page_evictable(page, NULL) - diverting these to the unevictable list.
651However, shrink_active_list() only sees unevictable pages that made it onto the 655However, shrink_active_list() only sees unevictable pages that made it onto the
652active/inactive lru lists. Note that these pages do not have PageUnevictable 656active/inactive lru lists. Note that these pages do not have PageUnevictable
653set - otherwise they would be on the unevictable list and shrink_active_list 657set - otherwise they would be on the unevictable list and shrink_active_list