aboutsummaryrefslogtreecommitdiffstats
path: root/Documentation
diff options
context:
space:
mode:
authorTom Zanussi <zanussi@us.ibm.com>2005-09-06 18:16:30 -0400
committerLinus Torvalds <torvalds@g5.osdl.org>2005-09-07 19:57:18 -0400
commite82894f84dbba130ab46c97748c03647f8204f92 (patch)
treedbf20825db44037f0db5d0696d43457292c546c3 /Documentation
parent8446f1d391f3d27e6bf9c43d4cbcdac0ca720417 (diff)
[PATCH] relayfs
Here's the latest version of relayfs, against linux-2.6.11-mm2. I'm hoping you'll consider putting this version back into your tree - the previous rounds of comment seem to have shaken out all the API issues and the number of comments on the code itself have also steadily dwindled. This patch is essentially the same as the relayfs redux part 5 patch, with some minor changes based on reviewer comments. Thanks again to Pekka Enberg for those. The patch size without documentation is now a little smaller at just over 40k. Here's a detailed list of the changes: - removed the attribute_flags in relay open and changed it to a boolean specifying either overwrite or no-overwrite mode, and removed everything referencing the attribute flags. - added a check for NULL names in relayfs_create_entry() - got rid of the unnecessary multiple labels in relay_create_buf() - some minor simplification of relay_alloc_buf() which got rid of a couple params - updated the Documentation In addition, this version (through code contained in the relay-apps tarball linked to below, not as part of the relayfs patch) tries to make it as easy as possible to create the cooperating kernel/user pieces of a typical and common type of logging application, one where kernel logging is kicked off when a user space data collection app starts and stops when the collection app exits, with the data being automatically logged to disk in between. To create this type of application, you basically just include a header file (relay-app.h, included in the relay-apps tarball) in your kernel module, define a couple of callbacks and call an initialization function, and on the user side call a single function that sets up and continuously monitors the buffers, and writes data to files as it becomes available. Channels are created when the collection app is started and destroyed when it exits, not when the kernel module is inserted, so different channel buffer sizes can be specified for each separate run via command-line options. See the README in the relay-apps tarball for details. Also included in the relay-apps tarball are a couple examples demonstrating how you can use this to create quick and dirty kernel logging/debugging applications. They are: - tprintk, short for 'tee printk', which temporarily puts a kprobe on printk() and writes a duplicate stream of printk output to a relayfs channel. This could be used anywhere there's printk() debugging code in the kernel which you'd like to exercise, but would rather not have your system logs cluttered with debugging junk. You'd probably want to kill klogd while you do this, otherwise there wouldn't be much point (since putting a kprobe on printk() doesn't change the output of printk()). I've used this method to temporarily divert the packet logging output of the iptables LOG target from the system logs to relayfs files instead, for instance. - klog, which just provides a printk-like formatted logging function on top of relayfs. Again, you can use this to keep stuff out of your system logs if used in place of printk. The example applications can be found here: http://prdownloads.sourceforge.net/dprobes/relay-apps.tar.gz?download From: Christoph Hellwig <hch@lst.de> avoid lookup_hash usage in relayfs Signed-off-by: Tom Zanussi <zanussi@us.ibm.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
Diffstat (limited to 'Documentation')
-rw-r--r--Documentation/filesystems/relayfs.txt362
1 files changed, 362 insertions, 0 deletions
diff --git a/Documentation/filesystems/relayfs.txt b/Documentation/filesystems/relayfs.txt
new file mode 100644
index 000000000000..d24e1b0d4f39
--- /dev/null
+++ b/Documentation/filesystems/relayfs.txt
@@ -0,0 +1,362 @@
1
2relayfs - a high-speed data relay filesystem
3============================================
4
5relayfs is a filesystem designed to provide an efficient mechanism for
6tools and facilities to relay large and potentially sustained streams
7of data from kernel space to user space.
8
9The main abstraction of relayfs is the 'channel'. A channel consists
10of a set of per-cpu kernel buffers each represented by a file in the
11relayfs filesystem. Kernel clients write into a channel using
12efficient write functions which automatically log to the current cpu's
13channel buffer. User space applications mmap() the per-cpu files and
14retrieve the data as it becomes available.
15
16The format of the data logged into the channel buffers is completely
17up to the relayfs client; relayfs does however provide hooks which
18allow clients to impose some stucture on the buffer data. Nor does
19relayfs implement any form of data filtering - this also is left to
20the client. The purpose is to keep relayfs as simple as possible.
21
22This document provides an overview of the relayfs API. The details of
23the function parameters are documented along with the functions in the
24filesystem code - please see that for details.
25
26Semantics
27=========
28
29Each relayfs channel has one buffer per CPU, each buffer has one or
30more sub-buffers. Messages are written to the first sub-buffer until
31it is too full to contain a new message, in which case it it is
32written to the next (if available). Messages are never split across
33sub-buffers. At this point, userspace can be notified so it empties
34the first sub-buffer, while the kernel continues writing to the next.
35
36When notified that a sub-buffer is full, the kernel knows how many
37bytes of it are padding i.e. unused. Userspace can use this knowledge
38to copy only valid data.
39
40After copying it, userspace can notify the kernel that a sub-buffer
41has been consumed.
42
43relayfs can operate in a mode where it will overwrite data not yet
44collected by userspace, and not wait for it to consume it.
45
46relayfs itself does not provide for communication of such data between
47userspace and kernel, allowing the kernel side to remain simple and not
48impose a single interface on userspace. It does provide a separate
49helper though, described below.
50
51klog, relay-app & librelay
52==========================
53
54relayfs itself is ready to use, but to make things easier, two
55additional systems are provided. klog is a simple wrapper to make
56writing formatted text or raw data to a channel simpler, regardless of
57whether a channel to write into exists or not, or whether relayfs is
58compiled into the kernel or is configured as a module. relay-app is
59the kernel counterpart of userspace librelay.c, combined these two
60files provide glue to easily stream data to disk, without having to
61bother with housekeeping. klog and relay-app can be used together,
62with klog providing high-level logging functions to the kernel and
63relay-app taking care of kernel-user control and disk-logging chores.
64
65It is possible to use relayfs without relay-app & librelay, but you'll
66have to implement communication between userspace and kernel, allowing
67both to convey the state of buffers (full, empty, amount of padding).
68
69klog, relay-app and librelay can be found in the relay-apps tarball on
70http://relayfs.sourceforge.net
71
72The relayfs user space API
73==========================
74
75relayfs implements basic file operations for user space access to
76relayfs channel buffer data. Here are the file operations that are
77available and some comments regarding their behavior:
78
79open() enables user to open an _existing_ buffer.
80
81mmap() results in channel buffer being mapped into the caller's
82 memory space. Note that you can't do a partial mmap - you must
83 map the entire file, which is NRBUF * SUBBUFSIZE.
84
85read() read the contents of a channel buffer. The bytes read are
86 'consumed' by the reader i.e. they won't be available again
87 to subsequent reads. If the channel is being used in
88 no-overwrite mode (the default), it can be read at any time
89 even if there's an active kernel writer. If the channel is
90 being used in overwrite mode and there are active channel
91 writers, results may be unpredictable - users should make
92 sure that all logging to the channel has ended before using
93 read() with overwrite mode.
94
95poll() POLLIN/POLLRDNORM/POLLERR supported. User applications are
96 notified when sub-buffer boundaries are crossed.
97
98close() decrements the channel buffer's refcount. When the refcount
99 reaches 0 i.e. when no process or kernel client has the buffer
100 open, the channel buffer is freed.
101
102
103In order for a user application to make use of relayfs files, the
104relayfs filesystem must be mounted. For example,
105
106 mount -t relayfs relayfs /mnt/relay
107
108NOTE: relayfs doesn't need to be mounted for kernel clients to create
109 or use channels - it only needs to be mounted when user space
110 applications need access to the buffer data.
111
112
113The relayfs kernel API
114======================
115
116Here's a summary of the API relayfs provides to in-kernel clients:
117
118
119 channel management functions:
120
121 relay_open(base_filename, parent, subbuf_size, n_subbufs,
122 callbacks)
123 relay_close(chan)
124 relay_flush(chan)
125 relay_reset(chan)
126 relayfs_create_dir(name, parent)
127 relayfs_remove_dir(dentry)
128
129 channel management typically called on instigation of userspace:
130
131 relay_subbufs_consumed(chan, cpu, subbufs_consumed)
132
133 write functions:
134
135 relay_write(chan, data, length)
136 __relay_write(chan, data, length)
137 relay_reserve(chan, length)
138
139 callbacks:
140
141 subbuf_start(buf, subbuf, prev_subbuf, prev_padding)
142 buf_mapped(buf, filp)
143 buf_unmapped(buf, filp)
144
145 helper functions:
146
147 relay_buf_full(buf)
148 subbuf_start_reserve(buf, length)
149
150
151Creating a channel
152------------------
153
154relay_open() is used to create a channel, along with its per-cpu
155channel buffers. Each channel buffer will have an associated file
156created for it in the relayfs filesystem, which can be opened and
157mmapped from user space if desired. The files are named
158basename0...basenameN-1 where N is the number of online cpus, and by
159default will be created in the root of the filesystem. If you want a
160directory structure to contain your relayfs files, you can create it
161with relayfs_create_dir() and pass the parent directory to
162relay_open(). Clients are responsible for cleaning up any directory
163structure they create when the channel is closed - use
164relayfs_remove_dir() for that.
165
166The total size of each per-cpu buffer is calculated by multiplying the
167number of sub-buffers by the sub-buffer size passed into relay_open().
168The idea behind sub-buffers is that they're basically an extension of
169double-buffering to N buffers, and they also allow applications to
170easily implement random-access-on-buffer-boundary schemes, which can
171be important for some high-volume applications. The number and size
172of sub-buffers is completely dependent on the application and even for
173the same application, different conditions will warrant different
174values for these parameters at different times. Typically, the right
175values to use are best decided after some experimentation; in general,
176though, it's safe to assume that having only 1 sub-buffer is a bad
177idea - you're guaranteed to either overwrite data or lose events
178depending on the channel mode being used.
179
180Channel 'modes'
181---------------
182
183relayfs channels can be used in either of two modes - 'overwrite' or
184'no-overwrite'. The mode is entirely determined by the implementation
185of the subbuf_start() callback, as described below. In 'overwrite'
186mode, also known as 'flight recorder' mode, writes continuously cycle
187around the buffer and will never fail, but will unconditionally
188overwrite old data regardless of whether it's actually been consumed.
189In no-overwrite mode, writes will fail i.e. data will be lost, if the
190number of unconsumed sub-buffers equals the total number of
191sub-buffers in the channel. It should be clear that if there is no
192consumer or if the consumer can't consume sub-buffers fast enought,
193data will be lost in either case; the only difference is whether data
194is lost from the beginning or the end of a buffer.
195
196As explained above, a relayfs channel is made of up one or more
197per-cpu channel buffers, each implemented as a circular buffer
198subdivided into one or more sub-buffers. Messages are written into
199the current sub-buffer of the channel's current per-cpu buffer via the
200write functions described below. Whenever a message can't fit into
201the current sub-buffer, because there's no room left for it, the
202client is notified via the subbuf_start() callback that a switch to a
203new sub-buffer is about to occur. The client uses this callback to 1)
204initialize the next sub-buffer if appropriate 2) finalize the previous
205sub-buffer if appropriate and 3) return a boolean value indicating
206whether or not to actually go ahead with the sub-buffer switch.
207
208To implement 'no-overwrite' mode, the userspace client would provide
209an implementation of the subbuf_start() callback something like the
210following:
211
212static int subbuf_start(struct rchan_buf *buf,
213 void *subbuf,
214 void *prev_subbuf,
215 unsigned int prev_padding)
216{
217 if (prev_subbuf)
218 *((unsigned *)prev_subbuf) = prev_padding;
219
220 if (relay_buf_full(buf))
221 return 0;
222
223 subbuf_start_reserve(buf, sizeof(unsigned int));
224
225 return 1;
226}
227
228If the current buffer is full i.e. all sub-buffers remain unconsumed,
229the callback returns 0 to indicate that the buffer switch should not
230occur yet i.e. until the consumer has had a chance to read the current
231set of ready sub-buffers. For the relay_buf_full() function to make
232sense, the consumer is reponsible for notifying relayfs when
233sub-buffers have been consumed via relay_subbufs_consumed(). Any
234subsequent attempts to write into the buffer will again invoke the
235subbuf_start() callback with the same parameters; only when the
236consumer has consumed one or more of the ready sub-buffers will
237relay_buf_full() return 0, in which case the buffer switch can
238continue.
239
240The implementation of the subbuf_start() callback for 'overwrite' mode
241would be very similar:
242
243static int subbuf_start(struct rchan_buf *buf,
244 void *subbuf,
245 void *prev_subbuf,
246 unsigned int prev_padding)
247{
248 if (prev_subbuf)
249 *((unsigned *)prev_subbuf) = prev_padding;
250
251 subbuf_start_reserve(buf, sizeof(unsigned int));
252
253 return 1;
254}
255
256In this case, the relay_buf_full() check is meaningless and the
257callback always returns 1, causing the buffer switch to occur
258unconditionally. It's also meaningless for the client to use the
259relay_subbufs_consumed() function in this mode, as it's never
260consulted.
261
262The default subbuf_start() implementation, used if the client doesn't
263define any callbacks, or doesn't define the subbuf_start() callback,
264implements the simplest possible 'no-overwrite' mode i.e. it does
265nothing but return 0.
266
267Header information can be reserved at the beginning of each sub-buffer
268by calling the subbuf_start_reserve() helper function from within the
269subbuf_start() callback. This reserved area can be used to store
270whatever information the client wants. In the example above, room is
271reserved in each sub-buffer to store the padding count for that
272sub-buffer. This is filled in for the previous sub-buffer in the
273subbuf_start() implementation; the padding value for the previous
274sub-buffer is passed into the subbuf_start() callback along with a
275pointer to the previous sub-buffer, since the padding value isn't
276known until a sub-buffer is filled. The subbuf_start() callback is
277also called for the first sub-buffer when the channel is opened, to
278give the client a chance to reserve space in it. In this case the
279previous sub-buffer pointer passed into the callback will be NULL, so
280the client should check the value of the prev_subbuf pointer before
281writing into the previous sub-buffer.
282
283Writing to a channel
284--------------------
285
286kernel clients write data into the current cpu's channel buffer using
287relay_write() or __relay_write(). relay_write() is the main logging
288function - it uses local_irqsave() to protect the buffer and should be
289used if you might be logging from interrupt context. If you know
290you'll never be logging from interrupt context, you can use
291__relay_write(), which only disables preemption. These functions
292don't return a value, so you can't determine whether or not they
293failed - the assumption is that you wouldn't want to check a return
294value in the fast logging path anyway, and that they'll always succeed
295unless the buffer is full and no-overwrite mode is being used, in
296which case you can detect a failed write in the subbuf_start()
297callback by calling the relay_buf_full() helper function.
298
299relay_reserve() is used to reserve a slot in a channel buffer which
300can be written to later. This would typically be used in applications
301that need to write directly into a channel buffer without having to
302stage data in a temporary buffer beforehand. Because the actual write
303may not happen immediately after the slot is reserved, applications
304using relay_reserve() can keep a count of the number of bytes actually
305written, either in space reserved in the sub-buffers themselves or as
306a separate array. See the 'reserve' example in the relay-apps tarball
307at http://relayfs.sourceforge.net for an example of how this can be
308done. Because the write is under control of the client and is
309separated from the reserve, relay_reserve() doesn't protect the buffer
310at all - it's up to the client to provide the appropriate
311synchronization when using relay_reserve().
312
313Closing a channel
314-----------------
315
316The client calls relay_close() when it's finished using the channel.
317The channel and its associated buffers are destroyed when there are no
318longer any references to any of the channel buffers. relay_flush()
319forces a sub-buffer switch on all the channel buffers, and can be used
320to finalize and process the last sub-buffers before the channel is
321closed.
322
323Misc
324----
325
326Some applications may want to keep a channel around and re-use it
327rather than open and close a new channel for each use. relay_reset()
328can be used for this purpose - it resets a channel to its initial
329state without reallocating channel buffer memory or destroying
330existing mappings. It should however only be called when it's safe to
331do so i.e. when the channel isn't currently being written to.
332
333Finally, there are a couple of utility callbacks that can be used for
334different purposes. buf_mapped() is called whenever a channel buffer
335is mmapped from user space and buf_unmapped() is called when it's
336unmapped. The client can use this notification to trigger actions
337within the kernel application, such as enabling/disabling logging to
338the channel.
339
340
341Resources
342=========
343
344For news, example code, mailing list, etc. see the relayfs homepage:
345
346 http://relayfs.sourceforge.net
347
348
349Credits
350=======
351
352The ideas and specs for relayfs came about as a result of discussions
353on tracing involving the following:
354
355Michel Dagenais <michel.dagenais@polymtl.ca>
356Richard Moore <richardj_moore@uk.ibm.com>
357Bob Wisniewski <bob@watson.ibm.com>
358Karim Yaghmour <karim@opersys.com>
359Tom Zanussi <zanussi@us.ibm.com>
360
361Also thanks to Hubertus Franke for a lot of useful suggestions and bug
362reports.