aboutsummaryrefslogtreecommitdiffstats
path: root/include
diff options
context:
space:
mode:
authorDavid Howells <dhowells@redhat.com>2009-04-03 11:42:36 -0400
committerDavid Howells <dhowells@redhat.com>2009-04-03 11:42:36 -0400
commit2d6fff637037395cc946ef910a880b5fa67b5370 (patch)
treea369011a976d5faf4fe45cf237503078cbbfb9b4 /include
parent266cf658efcf6ac33541a46740f74f50c79d2b6b (diff)
FS-Cache: Add the FS-Cache netfs API and documentation
Add the API for a generic facility (FS-Cache) by which filesystems (such as AFS or NFS) may call on local caching capabilities without having to know anything about how the cache works, or even if there is a cache: +---------+ | | +--------------+ | NFS |--+ | | | | | +-->| CacheFS | +---------+ | +----------+ | | /dev/hda5 | | | | | +--------------+ +---------+ +-->| | | | | | |--+ | AFS |----->| FS-Cache | | | | |--+ +---------+ +-->| | | | | | | +--------------+ +---------+ | +----------+ | | | | | | +-->| CacheFiles | | ISOFS |--+ | /var/cache | | | +--------------+ +---------+ General documentation and documentation of the netfs specific API are provided in addition to the header files. As this patch stands, it is possible to build a filesystem against the facility and attempt to use it. All that will happen is that all requests will be immediately denied as if no cache is present. Further patches will implement the core of the facility. The facility will transfer requests from networking filesystems to appropriate caches if possible, or else gracefully deny them. If this facility is disabled in the kernel configuration, then all its operations will trivially reduce to nothing during compilation. WHY NOT I_MAPPING? ================== I have added my own API to implement caching rather than using i_mapping to do this for a number of reasons. These have been discussed a lot on the LKML and CacheFS mailing lists, but to summarise the basics: (1) Most filesystems don't do hole reportage. Holes in files are treated as blocks of zeros and can't be distinguished otherwise, making it difficult to distinguish blocks that have been read from the network and cached from those that haven't. (2) The backing inode must be fully populated before being exposed to userspace through the main inode because the VM/VFS goes directly to the backing inode and does not interrogate the front inode's VM ops. Therefore: (a) The backing inode must fit entirely within the cache. (b) All backed files currently open must fit entirely within the cache at the same time. (c) A working set of files in total larger than the cache may not be cached. (d) A file may not grow larger than the available space in the cache. (e) A file that's open and cached, and remotely grows larger than the cache is potentially stuffed. (3) Writes go to the backing filesystem, and can only be transferred to the network when the file is closed. (4) There's no record of what changes have been made, so the whole file must be written back. (5) The pages belong to the backing filesystem, and all metadata associated with that page are relevant only to the backing filesystem, and not anything stacked atop it. OVERVIEW ======== FS-Cache provides (or will provide) the following facilities: (1) Caches can be added / removed at any time, even whilst in use. (2) Adds a facility by which tags can be used to refer to caches, even if they're not available yet. (3) More than one cache can be used at once. Caches can be selected explicitly by use of tags. (4) The netfs is provided with an interface that allows either party to withdraw caching facilities from a file (required for (1)). (5) A netfs may annotate cache objects that belongs to it. This permits the storage of coherency maintenance data. (6) Cache objects will be pinnable and space reservations will be possible. (7) The interface to the netfs returns as few errors as possible, preferring rather to let the netfs remain oblivious. (8) Cookies are used to represent indices, files and other objects to the netfs. The simplest cookie is just a NULL pointer - indicating nothing cached there. (9) The netfs is allowed to propose - dynamically - any index hierarchy it desires, though it must be aware that the index search function is recursive, stack space is limited, and indices can only be children of indices. (10) Indices can be used to group files together to reduce key size and to make group invalidation easier. The use of indices may make lookup quicker, but that's cache dependent. (11) Data I/O is effectively done directly to and from the netfs's pages. The netfs indicates that page A is at index B of the data-file represented by cookie C, and that it should be read or written. The cache backend may or may not start I/O on that page, but if it does, a netfs callback will be invoked to indicate completion. The I/O may be either synchronous or asynchronous. (12) Cookies can be "retired" upon release. At this point FS-Cache will mark them as obsolete and the index hierarchy rooted at that point will get recycled. (13) The netfs provides a "match" function for index searches. In addition to saying whether a match was made or not, this can also specify that an entry should be updated or deleted. FS-Cache maintains a virtual index tree in which all indices, files, objects and pages are kept. Bits of this tree may actually reside in one or more caches. FSDEF | +------------------------------------+ | | NFS AFS | | +--------------------------+ +-----------+ | | | | homedir mirror afs.org redhat.com | | | +------------+ +---------------+ +----------+ | | | | | | 00001 00002 00007 00125 vol00001 vol00002 | | | | | +---+---+ +-----+ +---+ +------+------+ +-----+----+ | | | | | | | | | | | | | PG0 PG1 PG2 PG0 XATTR PG0 PG1 DIRENT DIRENT DIRENT R/W R/O Bak | | PG0 +-------+ | | 00001 00003 | +---+---+ | | | PG0 PG1 PG2 In the example above, two netfs's can be seen to be backed: NFS and AFS. These have different index hierarchies: (*) The NFS primary index will probably contain per-server indices. Each server index is indexed by NFS file handles to get data file objects. Each data file objects can have an array of pages, but may also have further child objects, such as extended attributes and directory entries. Extended attribute objects themselves have page-array contents. (*) The AFS primary index contains per-cell indices. Each cell index contains per-logical-volume indices. Each of volume index contains up to three indices for the read-write, read-only and backup mirrors of those volumes. Each of these contains vnode data file objects, each of which contains an array of pages. The very top index is the FS-Cache master index in which individual netfs's have entries. Any index object may reside in more than one cache, provided it only has index children. Any index with non-index object children will be assumed to only reside in one cache. The FS-Cache overview can be found in: Documentation/filesystems/caching/fscache.txt The netfs API to FS-Cache can be found in: Documentation/filesystems/caching/netfs-api.txt Signed-off-by: David Howells <dhowells@redhat.com> Acked-by: Steve Dickson <steved@redhat.com> Acked-by: Trond Myklebust <Trond.Myklebust@netapp.com> Acked-by: Al Viro <viro@zeniv.linux.org.uk> Tested-by: Daire Byrne <Daire.Byrne@framestore.com>
Diffstat (limited to 'include')
-rw-r--r--include/linux/fscache.h548
1 files changed, 548 insertions, 0 deletions
diff --git a/include/linux/fscache.h b/include/linux/fscache.h
new file mode 100644
index 000000000000..feb3b0e0af4d
--- /dev/null
+++ b/include/linux/fscache.h
@@ -0,0 +1,548 @@
1/* General filesystem caching interface
2 *
3 * Copyright (C) 2004-2007 Red Hat, Inc. All Rights Reserved.
4 * Written by David Howells (dhowells@redhat.com)
5 *
6 * This program is free software; you can redistribute it and/or
7 * modify it under the terms of the GNU General Public License
8 * as published by the Free Software Foundation; either version
9 * 2 of the License, or (at your option) any later version.
10 *
11 * NOTE!!! See:
12 *
13 * Documentation/filesystems/caching/netfs-api.txt
14 *
15 * for a description of the network filesystem interface declared here.
16 */
17
18#ifndef _LINUX_FSCACHE_H
19#define _LINUX_FSCACHE_H
20
21#include <linux/fs.h>
22#include <linux/list.h>
23#include <linux/pagemap.h>
24#include <linux/pagevec.h>
25
26#if defined(CONFIG_FSCACHE) || defined(CONFIG_FSCACHE_MODULE)
27#define fscache_available() (1)
28#define fscache_cookie_valid(cookie) (cookie)
29#else
30#define fscache_available() (0)
31#define fscache_cookie_valid(cookie) (0)
32#endif
33
34
35/*
36 * overload PG_private_2 to give us PG_fscache - this is used to indicate that
37 * a page is currently backed by a local disk cache
38 */
39#define PageFsCache(page) PagePrivate2((page))
40#define SetPageFsCache(page) SetPagePrivate2((page))
41#define ClearPageFsCache(page) ClearPagePrivate2((page))
42#define TestSetPageFsCache(page) TestSetPagePrivate2((page))
43#define TestClearPageFsCache(page) TestClearPagePrivate2((page))
44
45/* pattern used to fill dead space in an index entry */
46#define FSCACHE_INDEX_DEADFILL_PATTERN 0x79
47
48struct pagevec;
49struct fscache_cache_tag;
50struct fscache_cookie;
51struct fscache_netfs;
52
53typedef void (*fscache_rw_complete_t)(struct page *page,
54 void *context,
55 int error);
56
57/* result of index entry consultation */
58enum fscache_checkaux {
59 FSCACHE_CHECKAUX_OKAY, /* entry okay as is */
60 FSCACHE_CHECKAUX_NEEDS_UPDATE, /* entry requires update */
61 FSCACHE_CHECKAUX_OBSOLETE, /* entry requires deletion */
62};
63
64/*
65 * fscache cookie definition
66 */
67struct fscache_cookie_def {
68 /* name of cookie type */
69 char name[16];
70
71 /* cookie type */
72 uint8_t type;
73#define FSCACHE_COOKIE_TYPE_INDEX 0
74#define FSCACHE_COOKIE_TYPE_DATAFILE 1
75
76 /* select the cache into which to insert an entry in this index
77 * - optional
78 * - should return a cache identifier or NULL to cause the cache to be
79 * inherited from the parent if possible or the first cache picked
80 * for a non-index file if not
81 */
82 struct fscache_cache_tag *(*select_cache)(
83 const void *parent_netfs_data,
84 const void *cookie_netfs_data);
85
86 /* get an index key
87 * - should store the key data in the buffer
88 * - should return the amount of amount stored
89 * - not permitted to return an error
90 * - the netfs data from the cookie being used as the source is
91 * presented
92 */
93 uint16_t (*get_key)(const void *cookie_netfs_data,
94 void *buffer,
95 uint16_t bufmax);
96
97 /* get certain file attributes from the netfs data
98 * - this function can be absent for an index
99 * - not permitted to return an error
100 * - the netfs data from the cookie being used as the source is
101 * presented
102 */
103 void (*get_attr)(const void *cookie_netfs_data, uint64_t *size);
104
105 /* get the auxilliary data from netfs data
106 * - this function can be absent if the index carries no state data
107 * - should store the auxilliary data in the buffer
108 * - should return the amount of amount stored
109 * - not permitted to return an error
110 * - the netfs data from the cookie being used as the source is
111 * presented
112 */
113 uint16_t (*get_aux)(const void *cookie_netfs_data,
114 void *buffer,
115 uint16_t bufmax);
116
117 /* consult the netfs about the state of an object
118 * - this function can be absent if the index carries no state data
119 * - the netfs data from the cookie being used as the target is
120 * presented, as is the auxilliary data
121 */
122 enum fscache_checkaux (*check_aux)(void *cookie_netfs_data,
123 const void *data,
124 uint16_t datalen);
125
126 /* get an extra reference on a read context
127 * - this function can be absent if the completion function doesn't
128 * require a context
129 */
130 void (*get_context)(void *cookie_netfs_data, void *context);
131
132 /* release an extra reference on a read context
133 * - this function can be absent if the completion function doesn't
134 * require a context
135 */
136 void (*put_context)(void *cookie_netfs_data, void *context);
137
138 /* indicate pages that now have cache metadata retained
139 * - this function should mark the specified pages as now being cached
140 * - the pages will have been marked with PG_fscache before this is
141 * called, so this is optional
142 */
143 void (*mark_pages_cached)(void *cookie_netfs_data,
144 struct address_space *mapping,
145 struct pagevec *cached_pvec);
146
147 /* indicate the cookie is no longer cached
148 * - this function is called when the backing store currently caching
149 * a cookie is removed
150 * - the netfs should use this to clean up any markers indicating
151 * cached pages
152 * - this is mandatory for any object that may have data
153 */
154 void (*now_uncached)(void *cookie_netfs_data);
155};
156
157/*
158 * fscache cached network filesystem type
159 * - name, version and ops must be filled in before registration
160 * - all other fields will be set during registration
161 */
162struct fscache_netfs {
163 uint32_t version; /* indexing version */
164 const char *name; /* filesystem name */
165 struct fscache_cookie *primary_index;
166 struct list_head link; /* internal link */
167};
168
169/*
170 * slow-path functions for when there is actually caching available, and the
171 * netfs does actually have a valid token
172 * - these are not to be called directly
173 * - these are undefined symbols when FS-Cache is not configured and the
174 * optimiser takes care of not using them
175 */
176
177/**
178 * fscache_register_netfs - Register a filesystem as desiring caching services
179 * @netfs: The description of the filesystem
180 *
181 * Register a filesystem as desiring caching services if they're available.
182 *
183 * See Documentation/filesystems/caching/netfs-api.txt for a complete
184 * description.
185 */
186static inline
187int fscache_register_netfs(struct fscache_netfs *netfs)
188{
189 return 0;
190}
191
192/**
193 * fscache_unregister_netfs - Indicate that a filesystem no longer desires
194 * caching services
195 * @netfs: The description of the filesystem
196 *
197 * Indicate that a filesystem no longer desires caching services for the
198 * moment.
199 *
200 * See Documentation/filesystems/caching/netfs-api.txt for a complete
201 * description.
202 */
203static inline
204void fscache_unregister_netfs(struct fscache_netfs *netfs)
205{
206}
207
208/**
209 * fscache_lookup_cache_tag - Look up a cache tag
210 * @name: The name of the tag to search for
211 *
212 * Acquire a specific cache referral tag that can be used to select a specific
213 * cache in which to cache an index.
214 *
215 * See Documentation/filesystems/caching/netfs-api.txt for a complete
216 * description.
217 */
218static inline
219struct fscache_cache_tag *fscache_lookup_cache_tag(const char *name)
220{
221 return NULL;
222}
223
224/**
225 * fscache_release_cache_tag - Release a cache tag
226 * @tag: The tag to release
227 *
228 * Release a reference to a cache referral tag previously looked up.
229 *
230 * See Documentation/filesystems/caching/netfs-api.txt for a complete
231 * description.
232 */
233static inline
234void fscache_release_cache_tag(struct fscache_cache_tag *tag)
235{
236}
237
238/**
239 * fscache_acquire_cookie - Acquire a cookie to represent a cache object
240 * @parent: The cookie that's to be the parent of this one
241 * @def: A description of the cache object, including callback operations
242 * @netfs_data: An arbitrary piece of data to be kept in the cookie to
243 * represent the cache object to the netfs
244 *
245 * This function is used to inform FS-Cache about part of an index hierarchy
246 * that can be used to locate files. This is done by requesting a cookie for
247 * each index in the path to the file.
248 *
249 * See Documentation/filesystems/caching/netfs-api.txt for a complete
250 * description.
251 */
252static inline
253struct fscache_cookie *fscache_acquire_cookie(
254 struct fscache_cookie *parent,
255 const struct fscache_cookie_def *def,
256 void *netfs_data)
257{
258 return NULL;
259}
260
261/**
262 * fscache_relinquish_cookie - Return the cookie to the cache, maybe discarding
263 * it
264 * @cookie: The cookie being returned
265 * @retire: True if the cache object the cookie represents is to be discarded
266 *
267 * This function returns a cookie to the cache, forcibly discarding the
268 * associated cache object if retire is set to true.
269 *
270 * See Documentation/filesystems/caching/netfs-api.txt for a complete
271 * description.
272 */
273static inline
274void fscache_relinquish_cookie(struct fscache_cookie *cookie, int retire)
275{
276}
277
278/**
279 * fscache_update_cookie - Request that a cache object be updated
280 * @cookie: The cookie representing the cache object
281 *
282 * Request an update of the index data for the cache object associated with the
283 * cookie.
284 *
285 * See Documentation/filesystems/caching/netfs-api.txt for a complete
286 * description.
287 */
288static inline
289void fscache_update_cookie(struct fscache_cookie *cookie)
290{
291}
292
293/**
294 * fscache_pin_cookie - Pin a data-storage cache object in its cache
295 * @cookie: The cookie representing the cache object
296 *
297 * Permit data-storage cache objects to be pinned in the cache.
298 *
299 * See Documentation/filesystems/caching/netfs-api.txt for a complete
300 * description.
301 */
302static inline
303int fscache_pin_cookie(struct fscache_cookie *cookie)
304{
305 return -ENOBUFS;
306}
307
308/**
309 * fscache_pin_cookie - Unpin a data-storage cache object in its cache
310 * @cookie: The cookie representing the cache object
311 *
312 * Permit data-storage cache objects to be unpinned from the cache.
313 *
314 * See Documentation/filesystems/caching/netfs-api.txt for a complete
315 * description.
316 */
317static inline
318void fscache_unpin_cookie(struct fscache_cookie *cookie)
319{
320}
321
322/**
323 * fscache_attr_changed - Notify cache that an object's attributes changed
324 * @cookie: The cookie representing the cache object
325 *
326 * Send a notification to the cache indicating that an object's attributes have
327 * changed. This includes the data size. These attributes will be obtained
328 * through the get_attr() cookie definition op.
329 *
330 * See Documentation/filesystems/caching/netfs-api.txt for a complete
331 * description.
332 */
333static inline
334int fscache_attr_changed(struct fscache_cookie *cookie)
335{
336 return -ENOBUFS;
337}
338
339/**
340 * fscache_reserve_space - Reserve data space for a cached object
341 * @cookie: The cookie representing the cache object
342 * @i_size: The amount of space to be reserved
343 *
344 * Reserve an amount of space in the cache for the cache object attached to a
345 * cookie so that a write to that object within the space can always be
346 * honoured.
347 *
348 * See Documentation/filesystems/caching/netfs-api.txt for a complete
349 * description.
350 */
351static inline
352int fscache_reserve_space(struct fscache_cookie *cookie, loff_t size)
353{
354 return -ENOBUFS;
355}
356
357/**
358 * fscache_read_or_alloc_page - Read a page from the cache or allocate a block
359 * in which to store it
360 * @cookie: The cookie representing the cache object
361 * @page: The netfs page to fill if possible
362 * @end_io_func: The callback to invoke when and if the page is filled
363 * @context: An arbitrary piece of data to pass on to end_io_func()
364 * @gfp: The conditions under which memory allocation should be made
365 *
366 * Read a page from the cache, or if that's not possible make a potential
367 * one-block reservation in the cache into which the page may be stored once
368 * fetched from the server.
369 *
370 * If the page is not backed by the cache object, or if it there's some reason
371 * it can't be, -ENOBUFS will be returned and nothing more will be done for
372 * that page.
373 *
374 * Else, if that page is backed by the cache, a read will be initiated directly
375 * to the netfs's page and 0 will be returned by this function. The
376 * end_io_func() callback will be invoked when the operation terminates on a
377 * completion or failure. Note that the callback may be invoked before the
378 * return.
379 *
380 * Else, if the page is unbacked, -ENODATA is returned and a block may have
381 * been allocated in the cache.
382 *
383 * See Documentation/filesystems/caching/netfs-api.txt for a complete
384 * description.
385 */
386static inline
387int fscache_read_or_alloc_page(struct fscache_cookie *cookie,
388 struct page *page,
389 fscache_rw_complete_t end_io_func,
390 void *context,
391 gfp_t gfp)
392{
393 return -ENOBUFS;
394}
395
396/**
397 * fscache_read_or_alloc_pages - Read pages from the cache and/or allocate
398 * blocks in which to store them
399 * @cookie: The cookie representing the cache object
400 * @mapping: The netfs inode mapping to which the pages will be attached
401 * @pages: A list of potential netfs pages to be filled
402 * @end_io_func: The callback to invoke when and if each page is filled
403 * @context: An arbitrary piece of data to pass on to end_io_func()
404 * @gfp: The conditions under which memory allocation should be made
405 *
406 * Read a set of pages from the cache, or if that's not possible, attempt to
407 * make a potential one-block reservation for each page in the cache into which
408 * that page may be stored once fetched from the server.
409 *
410 * If some pages are not backed by the cache object, or if it there's some
411 * reason they can't be, -ENOBUFS will be returned and nothing more will be
412 * done for that pages.
413 *
414 * Else, if some of the pages are backed by the cache, a read will be initiated
415 * directly to the netfs's page and 0 will be returned by this function. The
416 * end_io_func() callback will be invoked when the operation terminates on a
417 * completion or failure. Note that the callback may be invoked before the
418 * return.
419 *
420 * Else, if a page is unbacked, -ENODATA is returned and a block may have
421 * been allocated in the cache.
422 *
423 * Because the function may want to return all of -ENOBUFS, -ENODATA and 0 in
424 * regard to different pages, the return values are prioritised in that order.
425 * Any pages submitted for reading are removed from the pages list.
426 *
427 * See Documentation/filesystems/caching/netfs-api.txt for a complete
428 * description.
429 */
430static inline
431int fscache_read_or_alloc_pages(struct fscache_cookie *cookie,
432 struct address_space *mapping,
433 struct list_head *pages,
434 unsigned *nr_pages,
435 fscache_rw_complete_t end_io_func,
436 void *context,
437 gfp_t gfp)
438{
439 return -ENOBUFS;
440}
441
442/**
443 * fscache_alloc_page - Allocate a block in which to store a page
444 * @cookie: The cookie representing the cache object
445 * @page: The netfs page to allocate a page for
446 * @gfp: The conditions under which memory allocation should be made
447 *
448 * Request Allocation a block in the cache in which to store a netfs page
449 * without retrieving any contents from the cache.
450 *
451 * If the page is not backed by a file then -ENOBUFS will be returned and
452 * nothing more will be done, and no reservation will be made.
453 *
454 * Else, a block will be allocated if one wasn't already, and 0 will be
455 * returned
456 *
457 * See Documentation/filesystems/caching/netfs-api.txt for a complete
458 * description.
459 */
460static inline
461int fscache_alloc_page(struct fscache_cookie *cookie,
462 struct page *page,
463 gfp_t gfp)
464{
465 return -ENOBUFS;
466}
467
468/**
469 * fscache_write_page - Request storage of a page in the cache
470 * @cookie: The cookie representing the cache object
471 * @page: The netfs page to store
472 * @gfp: The conditions under which memory allocation should be made
473 *
474 * Request the contents of the netfs page be written into the cache. This
475 * request may be ignored if no cache block is currently allocated, in which
476 * case it will return -ENOBUFS.
477 *
478 * If a cache block was already allocated, a write will be initiated and 0 will
479 * be returned. The PG_fscache_write page bit is set immediately and will then
480 * be cleared at the completion of the write to indicate the success or failure
481 * of the operation. Note that the completion may happen before the return.
482 *
483 * See Documentation/filesystems/caching/netfs-api.txt for a complete
484 * description.
485 */
486static inline
487int fscache_write_page(struct fscache_cookie *cookie,
488 struct page *page,
489 gfp_t gfp)
490{
491 return -ENOBUFS;
492}
493
494/**
495 * fscache_uncache_page - Indicate that caching is no longer required on a page
496 * @cookie: The cookie representing the cache object
497 * @page: The netfs page that was being cached.
498 *
499 * Tell the cache that we no longer want a page to be cached and that it should
500 * remove any knowledge of the netfs page it may have.
501 *
502 * Note that this cannot cancel any outstanding I/O operations between this
503 * page and the cache.
504 *
505 * See Documentation/filesystems/caching/netfs-api.txt for a complete
506 * description.
507 */
508static inline
509void fscache_uncache_page(struct fscache_cookie *cookie,
510 struct page *page)
511{
512}
513
514/**
515 * fscache_check_page_write - Ask if a page is being writing to the cache
516 * @cookie: The cookie representing the cache object
517 * @page: The netfs page that is being cached.
518 *
519 * Ask the cache if a page is being written to the cache.
520 *
521 * See Documentation/filesystems/caching/netfs-api.txt for a complete
522 * description.
523 */
524static inline
525bool fscache_check_page_write(struct fscache_cookie *cookie,
526 struct page *page)
527{
528 return false;
529}
530
531/**
532 * fscache_wait_on_page_write - Wait for a page to complete writing to the cache
533 * @cookie: The cookie representing the cache object
534 * @page: The netfs page that is being cached.
535 *
536 * Ask the cache to wake us up when a page is no longer being written to the
537 * cache.
538 *
539 * See Documentation/filesystems/caching/netfs-api.txt for a complete
540 * description.
541 */
542static inline
543void fscache_wait_on_page_write(struct fscache_cookie *cookie,
544 struct page *page)
545{
546}
547
548#endif /* _LINUX_FSCACHE_H */