aboutsummaryrefslogtreecommitdiffstats
path: root/Documentation/video4linux/v4l2-framework.txt
diff options
context:
space:
mode:
Diffstat (limited to 'Documentation/video4linux/v4l2-framework.txt')
-rw-r--r--Documentation/video4linux/v4l2-framework.txt520
1 files changed, 520 insertions, 0 deletions
diff --git a/Documentation/video4linux/v4l2-framework.txt b/Documentation/video4linux/v4l2-framework.txt
new file mode 100644
index 000000000000..eeae76c22a93
--- /dev/null
+++ b/Documentation/video4linux/v4l2-framework.txt
@@ -0,0 +1,520 @@
1Overview of the V4L2 driver framework
2=====================================
3
4This text documents the various structures provided by the V4L2 framework and
5their relationships.
6
7
8Introduction
9------------
10
11The V4L2 drivers tend to be very complex due to the complexity of the
12hardware: most devices have multiple ICs, export multiple device nodes in
13/dev, and create also non-V4L2 devices such as DVB, ALSA, FB, I2C and input
14(IR) devices.
15
16Especially the fact that V4L2 drivers have to setup supporting ICs to
17do audio/video muxing/encoding/decoding makes it more complex than most.
18Usually these ICs are connected to the main bridge driver through one or
19more I2C busses, but other busses can also be used. Such devices are
20called 'sub-devices'.
21
22For a long time the framework was limited to the video_device struct for
23creating V4L device nodes and video_buf for handling the video buffers
24(note that this document does not discuss the video_buf framework).
25
26This meant that all drivers had to do the setup of device instances and
27connecting to sub-devices themselves. Some of this is quite complicated
28to do right and many drivers never did do it correctly.
29
30There is also a lot of common code that could never be refactored due to
31the lack of a framework.
32
33So this framework sets up the basic building blocks that all drivers
34need and this same framework should make it much easier to refactor
35common code into utility functions shared by all drivers.
36
37
38Structure of a driver
39---------------------
40
41All drivers have the following structure:
42
431) A struct for each device instance containing the device state.
44
452) A way of initializing and commanding sub-devices (if any).
46
473) Creating V4L2 device nodes (/dev/videoX, /dev/vbiX, /dev/radioX and
48 /dev/vtxX) and keeping track of device-node specific data.
49
504) Filehandle-specific structs containing per-filehandle data.
51
52This is a rough schematic of how it all relates:
53
54 device instances
55 |
56 +-sub-device instances
57 |
58 \-V4L2 device nodes
59 |
60 \-filehandle instances
61
62
63Structure of the framework
64--------------------------
65
66The framework closely resembles the driver structure: it has a v4l2_device
67struct for the device instance data, a v4l2_subdev struct to refer to
68sub-device instances, the video_device struct stores V4L2 device node data
69and in the future a v4l2_fh struct will keep track of filehandle instances
70(this is not yet implemented).
71
72
73struct v4l2_device
74------------------
75
76Each device instance is represented by a struct v4l2_device (v4l2-device.h).
77Very simple devices can just allocate this struct, but most of the time you
78would embed this struct inside a larger struct.
79
80You must register the device instance:
81
82 v4l2_device_register(struct device *dev, struct v4l2_device *v4l2_dev);
83
84Registration will initialize the v4l2_device struct and link dev->driver_data
85to v4l2_dev. Registration will also set v4l2_dev->name to a value derived from
86dev (driver name followed by the bus_id, to be precise). You may change the
87name after registration if you want.
88
89The first 'dev' argument is normally the struct device pointer of a pci_dev,
90usb_device or platform_device.
91
92You unregister with:
93
94 v4l2_device_unregister(struct v4l2_device *v4l2_dev);
95
96Unregistering will also automatically unregister all subdevs from the device.
97
98Sometimes you need to iterate over all devices registered by a specific
99driver. This is usually the case if multiple device drivers use the same
100hardware. E.g. the ivtvfb driver is a framebuffer driver that uses the ivtv
101hardware. The same is true for alsa drivers for example.
102
103You can iterate over all registered devices as follows:
104
105static int callback(struct device *dev, void *p)
106{
107 struct v4l2_device *v4l2_dev = dev_get_drvdata(dev);
108
109 /* test if this device was inited */
110 if (v4l2_dev == NULL)
111 return 0;
112 ...
113 return 0;
114}
115
116int iterate(void *p)
117{
118 struct device_driver *drv;
119 int err;
120
121 /* Find driver 'ivtv' on the PCI bus.
122 pci_bus_type is a global. For USB busses use usb_bus_type. */
123 drv = driver_find("ivtv", &pci_bus_type);
124 /* iterate over all ivtv device instances */
125 err = driver_for_each_device(drv, NULL, p, callback);
126 put_driver(drv);
127 return err;
128}
129
130Sometimes you need to keep a running counter of the device instance. This is
131commonly used to map a device instance to an index of a module option array.
132
133The recommended approach is as follows:
134
135static atomic_t drv_instance = ATOMIC_INIT(0);
136
137static int __devinit drv_probe(struct pci_dev *dev,
138 const struct pci_device_id *pci_id)
139{
140 ...
141 state->instance = atomic_inc_return(&drv_instance) - 1;
142}
143
144
145struct v4l2_subdev
146------------------
147
148Many drivers need to communicate with sub-devices. These devices can do all
149sort of tasks, but most commonly they handle audio and/or video muxing,
150encoding or decoding. For webcams common sub-devices are sensors and camera
151controllers.
152
153Usually these are I2C devices, but not necessarily. In order to provide the
154driver with a consistent interface to these sub-devices the v4l2_subdev struct
155(v4l2-subdev.h) was created.
156
157Each sub-device driver must have a v4l2_subdev struct. This struct can be
158stand-alone for simple sub-devices or it might be embedded in a larger struct
159if more state information needs to be stored. Usually there is a low-level
160device struct (e.g. i2c_client) that contains the device data as setup
161by the kernel. It is recommended to store that pointer in the private
162data of v4l2_subdev using v4l2_set_subdevdata(). That makes it easy to go
163from a v4l2_subdev to the actual low-level bus-specific device data.
164
165You also need a way to go from the low-level struct to v4l2_subdev. For the
166common i2c_client struct the i2c_set_clientdata() call is used to store a
167v4l2_subdev pointer, for other busses you may have to use other methods.
168
169From the bridge driver perspective you load the sub-device module and somehow
170obtain the v4l2_subdev pointer. For i2c devices this is easy: you call
171i2c_get_clientdata(). For other busses something similar needs to be done.
172Helper functions exists for sub-devices on an I2C bus that do most of this
173tricky work for you.
174
175Each v4l2_subdev contains function pointers that sub-device drivers can
176implement (or leave NULL if it is not applicable). Since sub-devices can do
177so many different things and you do not want to end up with a huge ops struct
178of which only a handful of ops are commonly implemented, the function pointers
179are sorted according to category and each category has its own ops struct.
180
181The top-level ops struct contains pointers to the category ops structs, which
182may be NULL if the subdev driver does not support anything from that category.
183
184It looks like this:
185
186struct v4l2_subdev_core_ops {
187 int (*g_chip_ident)(struct v4l2_subdev *sd, struct v4l2_chip_ident *chip);
188 int (*log_status)(struct v4l2_subdev *sd);
189 int (*init)(struct v4l2_subdev *sd, u32 val);
190 ...
191};
192
193struct v4l2_subdev_tuner_ops {
194 ...
195};
196
197struct v4l2_subdev_audio_ops {
198 ...
199};
200
201struct v4l2_subdev_video_ops {
202 ...
203};
204
205struct v4l2_subdev_ops {
206 const struct v4l2_subdev_core_ops *core;
207 const struct v4l2_subdev_tuner_ops *tuner;
208 const struct v4l2_subdev_audio_ops *audio;
209 const struct v4l2_subdev_video_ops *video;
210};
211
212The core ops are common to all subdevs, the other categories are implemented
213depending on the sub-device. E.g. a video device is unlikely to support the
214audio ops and vice versa.
215
216This setup limits the number of function pointers while still making it easy
217to add new ops and categories.
218
219A sub-device driver initializes the v4l2_subdev struct using:
220
221 v4l2_subdev_init(subdev, &ops);
222
223Afterwards you need to initialize subdev->name with a unique name and set the
224module owner. This is done for you if you use the i2c helper functions.
225
226A device (bridge) driver needs to register the v4l2_subdev with the
227v4l2_device:
228
229 int err = v4l2_device_register_subdev(device, subdev);
230
231This can fail if the subdev module disappeared before it could be registered.
232After this function was called successfully the subdev->dev field points to
233the v4l2_device.
234
235You can unregister a sub-device using:
236
237 v4l2_device_unregister_subdev(subdev);
238
239Afterwards the subdev module can be unloaded and subdev->dev == NULL.
240
241You can call an ops function either directly:
242
243 err = subdev->ops->core->g_chip_ident(subdev, &chip);
244
245but it is better and easier to use this macro:
246
247 err = v4l2_subdev_call(subdev, core, g_chip_ident, &chip);
248
249The macro will to the right NULL pointer checks and returns -ENODEV if subdev
250is NULL, -ENOIOCTLCMD if either subdev->core or subdev->core->g_chip_ident is
251NULL, or the actual result of the subdev->ops->core->g_chip_ident ops.
252
253It is also possible to call all or a subset of the sub-devices:
254
255 v4l2_device_call_all(dev, 0, core, g_chip_ident, &chip);
256
257Any subdev that does not support this ops is skipped and error results are
258ignored. If you want to check for errors use this:
259
260 err = v4l2_device_call_until_err(dev, 0, core, g_chip_ident, &chip);
261
262Any error except -ENOIOCTLCMD will exit the loop with that error. If no
263errors (except -ENOIOCTLCMD) occured, then 0 is returned.
264
265The second argument to both calls is a group ID. If 0, then all subdevs are
266called. If non-zero, then only those whose group ID match that value will
267be called. Before a bridge driver registers a subdev it can set subdev->grp_id
268to whatever value it wants (it's 0 by default). This value is owned by the
269bridge driver and the sub-device driver will never modify or use it.
270
271The group ID gives the bridge driver more control how callbacks are called.
272For example, there may be multiple audio chips on a board, each capable of
273changing the volume. But usually only one will actually be used when the
274user want to change the volume. You can set the group ID for that subdev to
275e.g. AUDIO_CONTROLLER and specify that as the group ID value when calling
276v4l2_device_call_all(). That ensures that it will only go to the subdev
277that needs it.
278
279The advantage of using v4l2_subdev is that it is a generic struct and does
280not contain any knowledge about the underlying hardware. So a driver might
281contain several subdevs that use an I2C bus, but also a subdev that is
282controlled through GPIO pins. This distinction is only relevant when setting
283up the device, but once the subdev is registered it is completely transparent.
284
285
286I2C sub-device drivers
287----------------------
288
289Since these drivers are so common, special helper functions are available to
290ease the use of these drivers (v4l2-common.h).
291
292The recommended method of adding v4l2_subdev support to an I2C driver is to
293embed the v4l2_subdev struct into the state struct that is created for each
294I2C device instance. Very simple devices have no state struct and in that case
295you can just create a v4l2_subdev directly.
296
297A typical state struct would look like this (where 'chipname' is replaced by
298the name of the chip):
299
300struct chipname_state {
301 struct v4l2_subdev sd;
302 ... /* additional state fields */
303};
304
305Initialize the v4l2_subdev struct as follows:
306
307 v4l2_i2c_subdev_init(&state->sd, client, subdev_ops);
308
309This function will fill in all the fields of v4l2_subdev and ensure that the
310v4l2_subdev and i2c_client both point to one another.
311
312You should also add a helper inline function to go from a v4l2_subdev pointer
313to a chipname_state struct:
314
315static inline struct chipname_state *to_state(struct v4l2_subdev *sd)
316{
317 return container_of(sd, struct chipname_state, sd);
318}
319
320Use this to go from the v4l2_subdev struct to the i2c_client struct:
321
322 struct i2c_client *client = v4l2_get_subdevdata(sd);
323
324And this to go from an i2c_client to a v4l2_subdev struct:
325
326 struct v4l2_subdev *sd = i2c_get_clientdata(client);
327
328Finally you need to make a command function to make driver->command()
329call the right subdev_ops functions:
330
331static int subdev_command(struct i2c_client *client, unsigned cmd, void *arg)
332{
333 return v4l2_subdev_command(i2c_get_clientdata(client), cmd, arg);
334}
335
336If driver->command is never used then you can leave this out. Eventually the
337driver->command usage should be removed from v4l.
338
339Make sure to call v4l2_device_unregister_subdev(sd) when the remove() callback
340is called. This will unregister the sub-device from the bridge driver. It is
341safe to call this even if the sub-device was never registered.
342
343
344The bridge driver also has some helper functions it can use:
345
346struct v4l2_subdev *sd = v4l2_i2c_new_subdev(adapter, "module_foo", "chipid", 0x36);
347
348This loads the given module (can be NULL if no module needs to be loaded) and
349calls i2c_new_device() with the given i2c_adapter and chip/address arguments.
350If all goes well, then it registers the subdev with the v4l2_device. It gets
351the v4l2_device by calling i2c_get_adapdata(adapter), so you should make sure
352that adapdata is set to v4l2_device when you setup the i2c_adapter in your
353driver.
354
355You can also use v4l2_i2c_new_probed_subdev() which is very similar to
356v4l2_i2c_new_subdev(), except that it has an array of possible I2C addresses
357that it should probe. Internally it calls i2c_new_probed_device().
358
359Both functions return NULL if something went wrong.
360
361
362struct video_device
363-------------------
364
365The actual device nodes in the /dev directory are created using the
366video_device struct (v4l2-dev.h). This struct can either be allocated
367dynamically or embedded in a larger struct.
368
369To allocate it dynamically use:
370
371 struct video_device *vdev = video_device_alloc();
372
373 if (vdev == NULL)
374 return -ENOMEM;
375
376 vdev->release = video_device_release;
377
378If you embed it in a larger struct, then you must set the release()
379callback to your own function:
380
381 struct video_device *vdev = &my_vdev->vdev;
382
383 vdev->release = my_vdev_release;
384
385The release callback must be set and it is called when the last user
386of the video device exits.
387
388The default video_device_release() callback just calls kfree to free the
389allocated memory.
390
391You should also set these fields:
392
393- parent: set to the parent device (same device as was used to register
394 v4l2_device).
395- name: set to something descriptive and unique.
396- fops: set to the file_operations struct.
397- ioctl_ops: if you use the v4l2_ioctl_ops to simplify ioctl maintenance
398 (highly recommended to use this and it might become compulsory in the
399 future!), then set this to your v4l2_ioctl_ops struct.
400
401If you use v4l2_ioctl_ops, then you should set .unlocked_ioctl to
402__video_ioctl2 or .ioctl to video_ioctl2 in your file_operations struct.
403
404
405video_device registration
406-------------------------
407
408Next you register the video device: this will create the character device
409for you.
410
411 err = video_register_device(vdev, VFL_TYPE_GRABBER, -1);
412 if (err) {
413 video_device_release(vdev); // or kfree(my_vdev);
414 return err;
415 }
416
417Which device is registered depends on the type argument. The following
418types exist:
419
420VFL_TYPE_GRABBER: videoX for video input/output devices
421VFL_TYPE_VBI: vbiX for vertical blank data (i.e. closed captions, teletext)
422VFL_TYPE_RADIO: radioX for radio tuners
423VFL_TYPE_VTX: vtxX for teletext devices (deprecated, don't use)
424
425The last argument gives you a certain amount of control over the device
426kernel number used (i.e. the X in videoX). Normally you will pass -1 to
427let the v4l2 framework pick the first free number. But if a driver creates
428many devices, then it can be useful to have different video devices in
429separate ranges. For example, video capture devices start at 0, video
430output devices start at 16.
431
432So you can use the last argument to specify a minimum kernel number and
433the v4l2 framework will try to pick the first free number that is equal
434or higher to what you passed. If that fails, then it will just pick the
435first free number.
436
437Whenever a device node is created some attributes are also created for you.
438If you look in /sys/class/video4linux you see the devices. Go into e.g.
439video0 and you will see 'name' and 'index' attributes. The 'name' attribute
440is the 'name' field of the video_device struct. The 'index' attribute is
441a device node index that can be assigned by the driver, or that is calculated
442for you.
443
444If you call video_register_device(), then the index is just increased by
4451 for each device node you register. The first video device node you register
446always starts off with 0.
447
448Alternatively you can call video_register_device_index() which is identical
449to video_register_device(), but with an extra index argument. Here you can
450pass a specific index value (between 0 and 31) that should be used.
451
452Users can setup udev rules that utilize the index attribute to make fancy
453device names (e.g. 'mpegX' for MPEG video capture device nodes).
454
455After the device was successfully registered, then you can use these fields:
456
457- vfl_type: the device type passed to video_register_device.
458- minor: the assigned device minor number.
459- num: the device kernel number (i.e. the X in videoX).
460- index: the device index number (calculated or set explicitly using
461 video_register_device_index).
462
463If the registration failed, then you need to call video_device_release()
464to free the allocated video_device struct, or free your own struct if the
465video_device was embedded in it. The vdev->release() callback will never
466be called if the registration failed, nor should you ever attempt to
467unregister the device if the registration failed.
468
469
470video_device cleanup
471--------------------
472
473When the video device nodes have to be removed, either during the unload
474of the driver or because the USB device was disconnected, then you should
475unregister them:
476
477 video_unregister_device(vdev);
478
479This will remove the device nodes from sysfs (causing udev to remove them
480from /dev).
481
482After video_unregister_device() returns no new opens can be done.
483
484However, in the case of USB devices some application might still have one
485of these device nodes open. You should block all new accesses to read,
486write, poll, etc. except possibly for certain ioctl operations like
487queueing buffers.
488
489When the last user of the video device node exits, then the vdev->release()
490callback is called and you can do the final cleanup there.
491
492
493video_device helper functions
494-----------------------------
495
496There are a few useful helper functions:
497
498You can set/get driver private data in the video_device struct using:
499
500void *video_get_drvdata(struct video_device *dev);
501void video_set_drvdata(struct video_device *dev, void *data);
502
503Note that you can safely call video_set_drvdata() before calling
504video_register_device().
505
506And this function:
507
508struct video_device *video_devdata(struct file *file);
509
510returns the video_device belonging to the file struct.
511
512The final helper function combines video_get_drvdata with
513video_devdata:
514
515void *video_drvdata(struct file *file);
516
517You can go from a video_device struct to the v4l2_device struct using:
518
519struct v4l2_device *v4l2_dev = dev_get_drvdata(vdev->parent);
520