diff options
-rw-r--r-- | Documentation/input/multi-touch-protocol.txt | 218 |
1 files changed, 149 insertions, 69 deletions
diff --git a/Documentation/input/multi-touch-protocol.txt b/Documentation/input/multi-touch-protocol.txt index c0fc1c75fd88..bdcba154b83e 100644 --- a/Documentation/input/multi-touch-protocol.txt +++ b/Documentation/input/multi-touch-protocol.txt | |||
@@ -6,31 +6,149 @@ Multi-touch (MT) Protocol | |||
6 | Introduction | 6 | Introduction |
7 | ------------ | 7 | ------------ |
8 | 8 | ||
9 | In order to utilize the full power of the new multi-touch devices, a way to | 9 | In order to utilize the full power of the new multi-touch and multi-user |
10 | report detailed finger data to user space is needed. This document | 10 | devices, a way to report detailed data from multiple contacts, i.e., |
11 | describes the multi-touch (MT) protocol which allows kernel drivers to | 11 | objects in direct contact with the device surface, is needed. This |
12 | report details for an arbitrary number of fingers. | 12 | document describes the multi-touch (MT) protocol which allows kernel |
13 | drivers to report details for an arbitrary number of contacts. | ||
14 | |||
15 | The protocol is divided into two types, depending on the capabilities of the | ||
16 | hardware. For devices handling anonymous contacts (type A), the protocol | ||
17 | describes how to send the raw data for all contacts to the receiver. For | ||
18 | devices capable of tracking identifiable contacts (type B), the protocol | ||
19 | describes how to send updates for individual contacts via event slots. | ||
20 | |||
21 | |||
22 | Protocol Usage | ||
23 | -------------- | ||
24 | |||
25 | Contact details are sent sequentially as separate packets of ABS_MT | ||
26 | events. Only the ABS_MT events are recognized as part of a contact | ||
27 | packet. Since these events are ignored by current single-touch (ST) | ||
28 | applications, the MT protocol can be implemented on top of the ST protocol | ||
29 | in an existing driver. | ||
30 | |||
31 | Drivers for type A devices separate contact packets by calling | ||
32 | input_mt_sync() at the end of each packet. This generates a SYN_MT_REPORT | ||
33 | event, which instructs the receiver to accept the data for the current | ||
34 | contact and prepare to receive another. | ||
35 | |||
36 | Drivers for type B devices separate contact packets by calling | ||
37 | input_mt_slot(), with a slot as argument, at the beginning of each packet. | ||
38 | This generates an ABS_MT_SLOT event, which instructs the receiver to | ||
39 | prepare for updates of the given slot. | ||
40 | |||
41 | All drivers mark the end of a multi-touch transfer by calling the usual | ||
42 | input_sync() function. This instructs the receiver to act upon events | ||
43 | accumulated since last EV_SYN/SYN_REPORT and prepare to receive a new set | ||
44 | of events/packets. | ||
45 | |||
46 | The main difference between the stateless type A protocol and the stateful | ||
47 | type B slot protocol lies in the usage of identifiable contacts to reduce | ||
48 | the amount of data sent to userspace. The slot protocol requires the use of | ||
49 | the ABS_MT_TRACKING_ID, either provided by the hardware or computed from | ||
50 | the raw data [5]. | ||
51 | |||
52 | For type A devices, the kernel driver should generate an arbitrary | ||
53 | enumeration of the full set of anonymous contacts currently on the | ||
54 | surface. The order in which the packets appear in the event stream is not | ||
55 | important. Event filtering and finger tracking is left to user space [3]. | ||
56 | |||
57 | For type B devices, the kernel driver should associate a slot with each | ||
58 | identified contact, and use that slot to propagate changes for the contact. | ||
59 | Creation, replacement and destruction of contacts is achieved by modifying | ||
60 | the ABS_MT_TRACKING_ID of the associated slot. A non-negative tracking id | ||
61 | is interpreted as a contact, and the value -1 denotes an unused slot. A | ||
62 | tracking id not previously present is considered new, and a tracking id no | ||
63 | longer present is considered removed. Since only changes are propagated, | ||
64 | the full state of each initiated contact has to reside in the receiving | ||
65 | end. Upon receiving an MT event, one simply updates the appropriate | ||
66 | attribute of the current slot. | ||
67 | |||
68 | |||
69 | Protocol Example A | ||
70 | ------------------ | ||
71 | |||
72 | Here is what a minimal event sequence for a two-contact touch would look | ||
73 | like for a type A device: | ||
74 | |||
75 | ABS_MT_POSITION_X x[0] | ||
76 | ABS_MT_POSITION_Y y[0] | ||
77 | SYN_MT_REPORT | ||
78 | ABS_MT_POSITION_X x[1] | ||
79 | ABS_MT_POSITION_Y y[1] | ||
80 | SYN_MT_REPORT | ||
81 | SYN_REPORT | ||
13 | 82 | ||
83 | The sequence after moving one of the contacts looks exactly the same; the | ||
84 | raw data for all present contacts are sent between every synchronization | ||
85 | with SYN_REPORT. | ||
14 | 86 | ||
15 | Usage | 87 | Here is the sequence after lifting the first contact: |
16 | ----- | 88 | |
89 | ABS_MT_POSITION_X x[1] | ||
90 | ABS_MT_POSITION_Y y[1] | ||
91 | SYN_MT_REPORT | ||
92 | SYN_REPORT | ||
93 | |||
94 | And here is the sequence after lifting the second contact: | ||
95 | |||
96 | SYN_MT_REPORT | ||
97 | SYN_REPORT | ||
98 | |||
99 | If the driver reports one of BTN_TOUCH or ABS_PRESSURE in addition to the | ||
100 | ABS_MT events, the last SYN_MT_REPORT event may be omitted. Otherwise, the | ||
101 | last SYN_REPORT will be dropped by the input core, resulting in no | ||
102 | zero-contact event reaching userland. | ||
17 | 103 | ||
18 | Anonymous finger details are sent sequentially as separate packets of ABS | 104 | |
19 | events. Only the ABS_MT events are recognized as part of a finger | 105 | Protocol Example B |
20 | packet. The end of a packet is marked by calling the input_mt_sync() | 106 | ------------------ |
21 | function, which generates a SYN_MT_REPORT event. This instructs the | 107 | |
22 | receiver to accept the data for the current finger and prepare to receive | 108 | Here is what a minimal event sequence for a two-contact touch would look |
23 | another. The end of a multi-touch transfer is marked by calling the usual | 109 | like for a type B device: |
24 | input_sync() function. This instructs the receiver to act upon events | 110 | |
25 | accumulated since last EV_SYN/SYN_REPORT and prepare to receive a new | 111 | ABS_MT_SLOT 0 |
26 | set of events/packets. | 112 | ABS_MT_TRACKING_ID 45 |
113 | ABS_MT_POSITION_X x[0] | ||
114 | ABS_MT_POSITION_Y y[0] | ||
115 | ABS_MT_SLOT 1 | ||
116 | ABS_MT_TRACKING_ID 46 | ||
117 | ABS_MT_POSITION_X x[1] | ||
118 | ABS_MT_POSITION_Y y[1] | ||
119 | SYN_REPORT | ||
120 | |||
121 | Here is the sequence after moving contact 45 in the x direction: | ||
122 | |||
123 | ABS_MT_SLOT 0 | ||
124 | ABS_MT_POSITION_X x[0] | ||
125 | SYN_REPORT | ||
126 | |||
127 | Here is the sequence after lifting the contact in slot 0: | ||
128 | |||
129 | ABS_MT_TRACKING_ID -1 | ||
130 | SYN_REPORT | ||
131 | |||
132 | The slot being modified is already 0, so the ABS_MT_SLOT is omitted. The | ||
133 | message removes the association of slot 0 with contact 45, thereby | ||
134 | destroying contact 45 and freeing slot 0 to be reused for another contact. | ||
135 | |||
136 | Finally, here is the sequence after lifting the second contact: | ||
137 | |||
138 | ABS_MT_SLOT 1 | ||
139 | ABS_MT_TRACKING_ID -1 | ||
140 | SYN_REPORT | ||
141 | |||
142 | |||
143 | Event Usage | ||
144 | ----------- | ||
27 | 145 | ||
28 | A set of ABS_MT events with the desired properties is defined. The events | 146 | A set of ABS_MT events with the desired properties is defined. The events |
29 | are divided into categories, to allow for partial implementation. The | 147 | are divided into categories, to allow for partial implementation. The |
30 | minimum set consists of ABS_MT_POSITION_X and ABS_MT_POSITION_Y, which | 148 | minimum set consists of ABS_MT_POSITION_X and ABS_MT_POSITION_Y, which |
31 | allows for multiple fingers to be tracked. If the device supports it, the | 149 | allows for multiple contacts to be tracked. If the device supports it, the |
32 | ABS_MT_TOUCH_MAJOR and ABS_MT_WIDTH_MAJOR may be used to provide the size | 150 | ABS_MT_TOUCH_MAJOR and ABS_MT_WIDTH_MAJOR may be used to provide the size |
33 | of the contact area and approaching finger, respectively. | 151 | of the contact area and approaching contact, respectively. |
34 | 152 | ||
35 | The TOUCH and WIDTH parameters have a geometrical interpretation; imagine | 153 | The TOUCH and WIDTH parameters have a geometrical interpretation; imagine |
36 | looking through a window at someone gently holding a finger against the | 154 | looking through a window at someone gently holding a finger against the |
@@ -41,56 +159,26 @@ ABS_MT_TOUCH_MAJOR, the diameter of the outer region is | |||
41 | ABS_MT_WIDTH_MAJOR. Now imagine the person pressing the finger harder | 159 | ABS_MT_WIDTH_MAJOR. Now imagine the person pressing the finger harder |
42 | against the glass. The inner region will increase, and in general, the | 160 | against the glass. The inner region will increase, and in general, the |
43 | ratio ABS_MT_TOUCH_MAJOR / ABS_MT_WIDTH_MAJOR, which is always smaller than | 161 | ratio ABS_MT_TOUCH_MAJOR / ABS_MT_WIDTH_MAJOR, which is always smaller than |
44 | unity, is related to the finger pressure. For pressure-based devices, | 162 | unity, is related to the contact pressure. For pressure-based devices, |
45 | ABS_MT_PRESSURE may be used to provide the pressure on the contact area | 163 | ABS_MT_PRESSURE may be used to provide the pressure on the contact area |
46 | instead. | 164 | instead. |
47 | 165 | ||
48 | In addition to the MAJOR parameters, the oval shape of the finger can be | 166 | In addition to the MAJOR parameters, the oval shape of the contact can be |
49 | described by adding the MINOR parameters, such that MAJOR and MINOR are the | 167 | described by adding the MINOR parameters, such that MAJOR and MINOR are the |
50 | major and minor axis of an ellipse. Finally, the orientation of the oval | 168 | major and minor axis of an ellipse. Finally, the orientation of the oval |
51 | shape can be describe with the ORIENTATION parameter. | 169 | shape can be describe with the ORIENTATION parameter. |
52 | 170 | ||
53 | The ABS_MT_TOOL_TYPE may be used to specify whether the touching tool is a | 171 | The ABS_MT_TOOL_TYPE may be used to specify whether the touching tool is a |
54 | finger or a pen or something else. Devices with more granular information | 172 | contact or a pen or something else. Devices with more granular information |
55 | may specify general shapes as blobs, i.e., as a sequence of rectangular | 173 | may specify general shapes as blobs, i.e., as a sequence of rectangular |
56 | shapes grouped together by an ABS_MT_BLOB_ID. Finally, for the few devices | 174 | shapes grouped together by an ABS_MT_BLOB_ID. Finally, for the few devices |
57 | that currently support it, the ABS_MT_TRACKING_ID event may be used to | 175 | that currently support it, the ABS_MT_TRACKING_ID event may be used to |
58 | report finger tracking from hardware [5]. | 176 | report contact tracking from hardware [5]. |
59 | 177 | ||
60 | Here is what a minimal event sequence for a two-finger touch would look | ||
61 | like: | ||
62 | |||
63 | ABS_MT_POSITION_X | ||
64 | ABS_MT_POSITION_Y | ||
65 | SYN_MT_REPORT | ||
66 | ABS_MT_POSITION_X | ||
67 | ABS_MT_POSITION_Y | ||
68 | SYN_MT_REPORT | ||
69 | SYN_REPORT | ||
70 | |||
71 | Here is the sequence after lifting one of the fingers: | ||
72 | |||
73 | ABS_MT_POSITION_X | ||
74 | ABS_MT_POSITION_Y | ||
75 | SYN_MT_REPORT | ||
76 | SYN_REPORT | ||
77 | |||
78 | And here is the sequence after lifting the remaining finger: | ||
79 | |||
80 | SYN_MT_REPORT | ||
81 | SYN_REPORT | ||
82 | |||
83 | If the driver reports one of BTN_TOUCH or ABS_PRESSURE in addition to the | ||
84 | ABS_MT events, the last SYN_MT_REPORT event may be omitted. Otherwise, the | ||
85 | last SYN_REPORT will be dropped by the input core, resulting in no | ||
86 | zero-finger event reaching userland. | ||
87 | 178 | ||
88 | Event Semantics | 179 | Event Semantics |
89 | --------------- | 180 | --------------- |
90 | 181 | ||
91 | The word "contact" is used to describe a tool which is in direct contact | ||
92 | with the surface. A finger, a pen or a rubber all classify as contacts. | ||
93 | |||
94 | ABS_MT_TOUCH_MAJOR | 182 | ABS_MT_TOUCH_MAJOR |
95 | 183 | ||
96 | The length of the major axis of the contact. The length should be given in | 184 | The length of the major axis of the contact. The length should be given in |
@@ -157,15 +245,16 @@ MT_TOOL_PEN [2]. | |||
157 | ABS_MT_BLOB_ID | 245 | ABS_MT_BLOB_ID |
158 | 246 | ||
159 | The BLOB_ID groups several packets together into one arbitrarily shaped | 247 | The BLOB_ID groups several packets together into one arbitrarily shaped |
160 | contact. This is a low-level anonymous grouping, and should not be confused | 248 | contact. This is a low-level anonymous grouping for type A devices, and |
161 | with the high-level trackingID [5]. Most kernel drivers will not have blob | 249 | should not be confused with the high-level trackingID [5]. Most type A |
162 | capability, and can safely omit the event. | 250 | devices do not have blob capability, so drivers can safely omit this event. |
163 | 251 | ||
164 | ABS_MT_TRACKING_ID | 252 | ABS_MT_TRACKING_ID |
165 | 253 | ||
166 | The TRACKING_ID identifies an initiated contact throughout its life cycle | 254 | The TRACKING_ID identifies an initiated contact throughout its life cycle |
167 | [5]. There are currently only a few devices that support it, so this event | 255 | [5]. This event is mandatory for type B devices. The value range of the |
168 | should normally be omitted. | 256 | TRACKING_ID should be large enough to ensure unique identification of a |
257 | contact maintained over an extended period of time. | ||
169 | 258 | ||
170 | 259 | ||
171 | Event Computation | 260 | Event Computation |
@@ -192,20 +281,11 @@ finger along the X axis (1). | |||
192 | Finger Tracking | 281 | Finger Tracking |
193 | --------------- | 282 | --------------- |
194 | 283 | ||
195 | The kernel driver should generate an arbitrary enumeration of the set of | ||
196 | anonymous contacts currently on the surface. The order in which the packets | ||
197 | appear in the event stream is not important. | ||
198 | |||
199 | The process of finger tracking, i.e., to assign a unique trackingID to each | 284 | The process of finger tracking, i.e., to assign a unique trackingID to each |
200 | initiated contact on the surface, is left to user space; preferably the | 285 | initiated contact on the surface, is a Euclidian Bipartite Matching |
201 | multi-touch X driver [3]. In that driver, the trackingID stays the same and | 286 | problem. At each event synchronization, the set of actual contacts is |
202 | unique until the contact vanishes (when the finger leaves the surface). The | 287 | matched to the set of contacts from the previous synchronization. A full |
203 | problem of assigning a set of anonymous fingers to a set of identified | 288 | implementation can be found in [3]. |
204 | fingers is a euclidian bipartite matching problem at each event update, and | ||
205 | relies on a sufficiently rapid update rate. | ||
206 | |||
207 | There are a few devices that support trackingID in hardware. User space can | ||
208 | make use of these native identifiers to reduce bandwidth and cpu usage. | ||
209 | 289 | ||
210 | 290 | ||
211 | Gestures | 291 | Gestures |