diff options
-rw-r--r-- | Documentation/MSI-HOWTO.txt | 174 |
1 files changed, 91 insertions, 83 deletions
diff --git a/Documentation/MSI-HOWTO.txt b/Documentation/MSI-HOWTO.txt index 63edc5f847c4..3ec6c720b016 100644 --- a/Documentation/MSI-HOWTO.txt +++ b/Documentation/MSI-HOWTO.txt | |||
@@ -10,14 +10,22 @@ | |||
10 | This guide describes the basics of Message Signaled Interrupts (MSI), | 10 | This guide describes the basics of Message Signaled Interrupts (MSI), |
11 | the advantages of using MSI over traditional interrupt mechanisms, | 11 | the advantages of using MSI over traditional interrupt mechanisms, |
12 | and how to enable your driver to use MSI or MSI-X. Also included is | 12 | and how to enable your driver to use MSI or MSI-X. Also included is |
13 | a Frequently Asked Questions. | 13 | a Frequently Asked Questions (FAQ) section. |
14 | |||
15 | 1.1 Terminology | ||
16 | |||
17 | PCI devices can be single-function or multi-function. In either case, | ||
18 | when this text talks about enabling or disabling MSI on a "device | ||
19 | function," it is referring to one specific PCI device and function and | ||
20 | not to all functions on a PCI device (unless the PCI device has only | ||
21 | one function). | ||
14 | 22 | ||
15 | 2. Copyright 2003 Intel Corporation | 23 | 2. Copyright 2003 Intel Corporation |
16 | 24 | ||
17 | 3. What is MSI/MSI-X? | 25 | 3. What is MSI/MSI-X? |
18 | 26 | ||
19 | Message Signaled Interrupt (MSI), as described in the PCI Local Bus | 27 | Message Signaled Interrupt (MSI), as described in the PCI Local Bus |
20 | Specification Revision 2.3 or latest, is an optional feature, and a | 28 | Specification Revision 2.3 or later, is an optional feature, and a |
21 | required feature for PCI Express devices. MSI enables a device function | 29 | required feature for PCI Express devices. MSI enables a device function |
22 | to request service by sending an Inbound Memory Write on its PCI bus to | 30 | to request service by sending an Inbound Memory Write on its PCI bus to |
23 | the FSB as a Message Signal Interrupt transaction. Because MSI is | 31 | the FSB as a Message Signal Interrupt transaction. Because MSI is |
@@ -27,7 +35,7 @@ supported. | |||
27 | 35 | ||
28 | A PCI device that supports MSI must also support pin IRQ assertion | 36 | A PCI device that supports MSI must also support pin IRQ assertion |
29 | interrupt mechanism to provide backward compatibility for systems that | 37 | interrupt mechanism to provide backward compatibility for systems that |
30 | do not support MSI. In Systems, which support MSI, the bus driver is | 38 | do not support MSI. In systems which support MSI, the bus driver is |
31 | responsible for initializing the message address and message data of | 39 | responsible for initializing the message address and message data of |
32 | the device function's MSI/MSI-X capability structure during device | 40 | the device function's MSI/MSI-X capability structure during device |
33 | initial configuration. | 41 | initial configuration. |
@@ -61,17 +69,17 @@ over the MSI capability structure as described below. | |||
61 | 69 | ||
62 | - MSI and MSI-X both support per-vector masking. Per-vector | 70 | - MSI and MSI-X both support per-vector masking. Per-vector |
63 | masking is an optional extension of MSI but a required | 71 | masking is an optional extension of MSI but a required |
64 | feature for MSI-X. Per-vector masking provides the kernel | 72 | feature for MSI-X. Per-vector masking provides the kernel the |
65 | the ability to mask/unmask MSI when servicing its software | 73 | ability to mask/unmask a single MSI while running its |
66 | interrupt service routing handler. If per-vector masking is | 74 | interrupt service routine. If per-vector masking is |
67 | not supported, then the device driver should provide the | 75 | not supported, then the device driver should provide the |
68 | hardware/software synchronization to ensure that the device | 76 | hardware/software synchronization to ensure that the device |
69 | generates MSI when the driver wants it to do so. | 77 | generates MSI when the driver wants it to do so. |
70 | 78 | ||
71 | 4. Why use MSI? | 79 | 4. Why use MSI? |
72 | 80 | ||
73 | As a benefit the simplification of board design, MSI allows board | 81 | As a benefit to the simplification of board design, MSI allows board |
74 | designers to remove out of band interrupt routing. MSI is another | 82 | designers to remove out-of-band interrupt routing. MSI is another |
75 | step towards a legacy-free environment. | 83 | step towards a legacy-free environment. |
76 | 84 | ||
77 | Due to increasing pressure on chipset and processor packages to | 85 | Due to increasing pressure on chipset and processor packages to |
@@ -87,7 +95,7 @@ support. As a result, the PCI Express technology requires MSI | |||
87 | support for better interrupt performance. | 95 | support for better interrupt performance. |
88 | 96 | ||
89 | Using MSI enables the device functions to support two or more | 97 | Using MSI enables the device functions to support two or more |
90 | vectors, which can be configured to target different CPU's to | 98 | vectors, which can be configured to target different CPUs to |
91 | increase scalability. | 99 | increase scalability. |
92 | 100 | ||
93 | 5. Configuring a driver to use MSI/MSI-X | 101 | 5. Configuring a driver to use MSI/MSI-X |
@@ -119,13 +127,13 @@ pci_enable_msi() explicitly. | |||
119 | 127 | ||
120 | int pci_enable_msi(struct pci_dev *dev) | 128 | int pci_enable_msi(struct pci_dev *dev) |
121 | 129 | ||
122 | With this new API, any existing device driver, which like to have | 130 | With this new API, a device driver that wants to have MSI |
123 | MSI enabled on its device function, must call this API to enable MSI | 131 | enabled on its device function must call this API to enable MSI. |
124 | A successful call will initialize the MSI capability structure | 132 | A successful call will initialize the MSI capability structure |
125 | with ONE vector, regardless of whether a device function is | 133 | with ONE vector, regardless of whether a device function is |
126 | capable of supporting multiple messages. This vector replaces the | 134 | capable of supporting multiple messages. This vector replaces the |
127 | pre-assigned dev->irq with a new MSI vector. To avoid the conflict | 135 | pre-assigned dev->irq with a new MSI vector. To avoid a conflict |
128 | of new assigned vector with existing pre-assigned vector requires | 136 | of the new assigned vector with existing pre-assigned vector requires |
129 | a device driver to call this API before calling request_irq(). | 137 | a device driver to call this API before calling request_irq(). |
130 | 138 | ||
131 | 5.2.2 API pci_disable_msi | 139 | 5.2.2 API pci_disable_msi |
@@ -137,14 +145,14 @@ when a device driver is unloading. This API restores dev->irq with | |||
137 | the pre-assigned IOAPIC vector and switches a device's interrupt | 145 | the pre-assigned IOAPIC vector and switches a device's interrupt |
138 | mode to PCI pin-irq assertion/INTx emulation mode. | 146 | mode to PCI pin-irq assertion/INTx emulation mode. |
139 | 147 | ||
140 | Note that a device driver should always call free_irq() on MSI vector | 148 | Note that a device driver should always call free_irq() on the MSI vector |
141 | it has done request_irq() on before calling this API. Failure to do | 149 | that it has done request_irq() on before calling this API. Failure to do |
142 | so results a BUG_ON() and a device will be left with MSI enabled and | 150 | so results in a BUG_ON() and a device will be left with MSI enabled and |
143 | leaks its vector. | 151 | leaks its vector. |
144 | 152 | ||
145 | 5.2.3 MSI mode vs. legacy mode diagram | 153 | 5.2.3 MSI mode vs. legacy mode diagram |
146 | 154 | ||
147 | The below diagram shows the events, which switches the interrupt | 155 | The below diagram shows the events which switch the interrupt |
148 | mode on the MSI-capable device function between MSI mode and | 156 | mode on the MSI-capable device function between MSI mode and |
149 | PIN-IRQ assertion mode. | 157 | PIN-IRQ assertion mode. |
150 | 158 | ||
@@ -155,9 +163,9 @@ PIN-IRQ assertion mode. | |||
155 | ------------ pci_disable_msi ------------------------ | 163 | ------------ pci_disable_msi ------------------------ |
156 | 164 | ||
157 | 165 | ||
158 | Figure 1.0 MSI Mode vs. Legacy Mode | 166 | Figure 1. MSI Mode vs. Legacy Mode |
159 | 167 | ||
160 | In Figure 1.0, a device operates by default in legacy mode. Legacy | 168 | In Figure 1, a device operates by default in legacy mode. Legacy |
161 | in this context means PCI pin-irq assertion or PCI-Express INTx | 169 | in this context means PCI pin-irq assertion or PCI-Express INTx |
162 | emulation. A successful MSI request (using pci_enable_msi()) switches | 170 | emulation. A successful MSI request (using pci_enable_msi()) switches |
163 | a device's interrupt mode to MSI mode. A pre-assigned IOAPIC vector | 171 | a device's interrupt mode to MSI mode. A pre-assigned IOAPIC vector |
@@ -166,11 +174,11 @@ assigned MSI vector will replace dev->irq. | |||
166 | 174 | ||
167 | To return back to its default mode, a device driver should always call | 175 | To return back to its default mode, a device driver should always call |
168 | pci_disable_msi() to undo the effect of pci_enable_msi(). Note that a | 176 | pci_disable_msi() to undo the effect of pci_enable_msi(). Note that a |
169 | device driver should always call free_irq() on MSI vector it has done | 177 | device driver should always call free_irq() on the MSI vector it has |
170 | request_irq() on before calling pci_disable_msi(). Failure to do so | 178 | done request_irq() on before calling pci_disable_msi(). Failure to do |
171 | results a BUG_ON() and a device will be left with MSI enabled and | 179 | so results in a BUG_ON() and a device will be left with MSI enabled and |
172 | leaks its vector. Otherwise, the PCI subsystem restores a device's | 180 | leaks its vector. Otherwise, the PCI subsystem restores a device's |
173 | dev->irq with a pre-assigned IOAPIC vector and marks released | 181 | dev->irq with a pre-assigned IOAPIC vector and marks the released |
174 | MSI vector as unused. | 182 | MSI vector as unused. |
175 | 183 | ||
176 | Once being marked as unused, there is no guarantee that the PCI | 184 | Once being marked as unused, there is no guarantee that the PCI |
@@ -178,8 +186,8 @@ subsystem will reserve this MSI vector for a device. Depending on | |||
178 | the availability of current PCI vector resources and the number of | 186 | the availability of current PCI vector resources and the number of |
179 | MSI/MSI-X requests from other drivers, this MSI may be re-assigned. | 187 | MSI/MSI-X requests from other drivers, this MSI may be re-assigned. |
180 | 188 | ||
181 | For the case where the PCI subsystem re-assigned this MSI vector | 189 | For the case where the PCI subsystem re-assigns this MSI vector to |
182 | another driver, a request to switching back to MSI mode may result | 190 | another driver, a request to switch back to MSI mode may result |
183 | in being assigned a different MSI vector or a failure if no more | 191 | in being assigned a different MSI vector or a failure if no more |
184 | vectors are available. | 192 | vectors are available. |
185 | 193 | ||
@@ -208,12 +216,12 @@ Unlike the function pci_enable_msi(), the function pci_enable_msix() | |||
208 | does not replace the pre-assigned IOAPIC dev->irq with a new MSI | 216 | does not replace the pre-assigned IOAPIC dev->irq with a new MSI |
209 | vector because the PCI subsystem writes the 1:1 vector-to-entry mapping | 217 | vector because the PCI subsystem writes the 1:1 vector-to-entry mapping |
210 | into the field vector of each element contained in a second argument. | 218 | into the field vector of each element contained in a second argument. |
211 | Note that the pre-assigned IO-APIC dev->irq is valid only if the device | 219 | Note that the pre-assigned IOAPIC dev->irq is valid only if the device |
212 | operates in PIN-IRQ assertion mode. In MSI-X mode, any attempt of | 220 | operates in PIN-IRQ assertion mode. In MSI-X mode, any attempt at |
213 | using dev->irq by the device driver to request for interrupt service | 221 | using dev->irq by the device driver to request for interrupt service |
214 | may result unpredictabe behavior. | 222 | may result unpredictabe behavior. |
215 | 223 | ||
216 | For each MSI-X vector granted, a device driver is responsible to call | 224 | For each MSI-X vector granted, a device driver is responsible for calling |
217 | other functions like request_irq(), enable_irq(), etc. to enable | 225 | other functions like request_irq(), enable_irq(), etc. to enable |
218 | this vector with its corresponding interrupt service handler. It is | 226 | this vector with its corresponding interrupt service handler. It is |
219 | a device driver's choice to assign all vectors with the same | 227 | a device driver's choice to assign all vectors with the same |
@@ -224,13 +232,13 @@ service handler. | |||
224 | 232 | ||
225 | The PCI 3.0 specification has implementation notes that MMIO address | 233 | The PCI 3.0 specification has implementation notes that MMIO address |
226 | space for a device's MSI-X structure should be isolated so that the | 234 | space for a device's MSI-X structure should be isolated so that the |
227 | software system can set different page for controlling accesses to | 235 | software system can set different pages for controlling accesses to the |
228 | the MSI-X structure. The implementation of MSI patch requires the PCI | 236 | MSI-X structure. The implementation of MSI support requires the PCI |
229 | subsystem, not a device driver, to maintain full control of the MSI-X | 237 | subsystem, not a device driver, to maintain full control of the MSI-X |
230 | table/MSI-X PBA and MMIO address space of the MSI-X table/MSI-X PBA. | 238 | table/MSI-X PBA (Pending Bit Array) and MMIO address space of the MSI-X |
231 | A device driver is prohibited from requesting the MMIO address space | 239 | table/MSI-X PBA. A device driver is prohibited from requesting the MMIO |
232 | of the MSI-X table/MSI-X PBA. Otherwise, the PCI subsystem will fail | 240 | address space of the MSI-X table/MSI-X PBA. Otherwise, the PCI subsystem |
233 | enabling MSI-X on its hardware device when it calls the function | 241 | will fail enabling MSI-X on its hardware device when it calls the function |
234 | pci_enable_msix(). | 242 | pci_enable_msix(). |
235 | 243 | ||
236 | 5.3.2 Handling MSI-X allocation | 244 | 5.3.2 Handling MSI-X allocation |
@@ -274,9 +282,9 @@ For the case where fewer MSI-X vectors are allocated to a function | |||
274 | than requested, the function pci_enable_msix() will return the | 282 | than requested, the function pci_enable_msix() will return the |
275 | maximum number of MSI-X vectors available to the caller. A device | 283 | maximum number of MSI-X vectors available to the caller. A device |
276 | driver may re-send its request with fewer or equal vectors indicated | 284 | driver may re-send its request with fewer or equal vectors indicated |
277 | in a return. For example, if a device driver requests 5 vectors, but | 285 | in the return. For example, if a device driver requests 5 vectors, but |
278 | the number of available vectors is 3 vectors, a value of 3 will be a | 286 | the number of available vectors is 3 vectors, a value of 3 will be |
279 | return as a result of pci_enable_msix() call. A function could be | 287 | returned as a result of pci_enable_msix() call. A function could be |
280 | designed for its driver to use only 3 MSI-X table entries as | 288 | designed for its driver to use only 3 MSI-X table entries as |
281 | different combinations as ABC--, A-B-C, A--CB, etc. Note that this | 289 | different combinations as ABC--, A-B-C, A--CB, etc. Note that this |
282 | patch does not support multiple entries with the same vector. Such | 290 | patch does not support multiple entries with the same vector. Such |
@@ -285,49 +293,46 @@ as ABBCC, AABCC, BCCBA, etc will result as a failure by the function | |||
285 | pci_enable_msix(). Below are the reasons why supporting multiple | 293 | pci_enable_msix(). Below are the reasons why supporting multiple |
286 | entries with the same vector is an undesirable solution. | 294 | entries with the same vector is an undesirable solution. |
287 | 295 | ||
288 | - The PCI subsystem can not determine which entry, which | 296 | - The PCI subsystem cannot determine the entry that |
289 | generated the message, to mask/unmask MSI while handling | 297 | generated the message to mask/unmask MSI while handling |
290 | software driver ISR. Attempting to walk through all MSI-X | 298 | software driver ISR. Attempting to walk through all MSI-X |
291 | table entries (2048 max) to mask/unmask any match vector | 299 | table entries (2048 max) to mask/unmask any match vector |
292 | is an undesirable solution. | 300 | is an undesirable solution. |
293 | 301 | ||
294 | - Walk through all MSI-X table entries (2048 max) to handle | 302 | - Walking through all MSI-X table entries (2048 max) to handle |
295 | SMP affinity of any match vector is an undesirable solution. | 303 | SMP affinity of any match vector is an undesirable solution. |
296 | 304 | ||
297 | 5.3.4 API pci_enable_msix | 305 | 5.3.4 API pci_enable_msix |
298 | 306 | ||
299 | int pci_enable_msix(struct pci_dev *dev, u32 *entries, int nvec) | 307 | int pci_enable_msix(struct pci_dev *dev, struct msix_entry *entries, int nvec) |
300 | 308 | ||
301 | This API enables a device driver to request the PCI subsystem | 309 | This API enables a device driver to request the PCI subsystem |
302 | for enabling MSI-X messages on its hardware device. Depending on | 310 | to enable MSI-X messages on its hardware device. Depending on |
303 | the availability of PCI vectors resources, the PCI subsystem enables | 311 | the availability of PCI vectors resources, the PCI subsystem enables |
304 | either all or nothing. | 312 | either all or none of the requested vectors. |
305 | 313 | ||
306 | Argument dev points to the device (pci_dev) structure. | 314 | Argument 'dev' points to the device (pci_dev) structure. |
307 | 315 | ||
308 | Argument entries is a pointer of unsigned integer type. The number of | 316 | Argument 'entries' is a pointer to an array of msix_entry structs. |
309 | elements is indicated in argument nvec. The content of each element | 317 | The number of entries is indicated in argument 'nvec'. |
310 | will be mapped to the following struct defined in /driver/pci/msi.h. | 318 | struct msix_entry is defined in /driver/pci/msi.h: |
311 | 319 | ||
312 | struct msix_entry { | 320 | struct msix_entry { |
313 | u16 vector; /* kernel uses to write alloc vector */ | 321 | u16 vector; /* kernel uses to write alloc vector */ |
314 | u16 entry; /* driver uses to specify entry */ | 322 | u16 entry; /* driver uses to specify entry */ |
315 | }; | 323 | }; |
316 | 324 | ||
317 | A device driver is responsible for initializing the field entry of | 325 | A device driver is responsible for initializing the field 'entry' of |
318 | each element with unique entry supported by MSI-X table. Otherwise, | 326 | each element with a unique entry supported by MSI-X table. Otherwise, |
319 | -EINVAL will be returned as a result. A successful return of zero | 327 | -EINVAL will be returned as a result. A successful return of zero |
320 | indicates the PCI subsystem completes initializing each of requested | 328 | indicates the PCI subsystem completed initializing each of the requested |
321 | entries of the MSI-X table with message address and message data. | 329 | entries of the MSI-X table with message address and message data. |
322 | Last but not least, the PCI subsystem will write the 1:1 | 330 | Last but not least, the PCI subsystem will write the 1:1 |
323 | vector-to-entry mapping into the field vector of each element. A | 331 | vector-to-entry mapping into the field 'vector' of each element. A |
324 | device driver is responsible of keeping track of allocated MSI-X | 332 | device driver is responsible for keeping track of allocated MSI-X |
325 | vectors in its internal data structure. | 333 | vectors in its internal data structure. |
326 | 334 | ||
327 | Argument nvec is an integer indicating the number of messages | 335 | A return of zero indicates that the number of MSI-X vectors was |
328 | requested. | ||
329 | |||
330 | A return of zero indicates that the number of MSI-X vectors is | ||
331 | successfully allocated. A return of greater than zero indicates | 336 | successfully allocated. A return of greater than zero indicates |
332 | MSI-X vector shortage. Or a return of less than zero indicates | 337 | MSI-X vector shortage. Or a return of less than zero indicates |
333 | a failure. This failure may be a result of duplicate entries | 338 | a failure. This failure may be a result of duplicate entries |
@@ -341,12 +346,12 @@ void pci_disable_msix(struct pci_dev *dev) | |||
341 | This API should always be used to undo the effect of pci_enable_msix() | 346 | This API should always be used to undo the effect of pci_enable_msix() |
342 | when a device driver is unloading. Note that a device driver should | 347 | when a device driver is unloading. Note that a device driver should |
343 | always call free_irq() on all MSI-X vectors it has done request_irq() | 348 | always call free_irq() on all MSI-X vectors it has done request_irq() |
344 | on before calling this API. Failure to do so results a BUG_ON() and | 349 | on before calling this API. Failure to do so results in a BUG_ON() and |
345 | a device will be left with MSI-X enabled and leaks its vectors. | 350 | a device will be left with MSI-X enabled and leaks its vectors. |
346 | 351 | ||
347 | 5.3.6 MSI-X mode vs. legacy mode diagram | 352 | 5.3.6 MSI-X mode vs. legacy mode diagram |
348 | 353 | ||
349 | The below diagram shows the events, which switches the interrupt | 354 | The below diagram shows the events which switch the interrupt |
350 | mode on the MSI-X capable device function between MSI-X mode and | 355 | mode on the MSI-X capable device function between MSI-X mode and |
351 | PIN-IRQ assertion mode (legacy). | 356 | PIN-IRQ assertion mode (legacy). |
352 | 357 | ||
@@ -356,22 +361,22 @@ PIN-IRQ assertion mode (legacy). | |||
356 | | | ===============> | | | 361 | | | ===============> | | |
357 | ------------ pci_disable_msix ------------------------ | 362 | ------------ pci_disable_msix ------------------------ |
358 | 363 | ||
359 | Figure 2.0 MSI-X Mode vs. Legacy Mode | 364 | Figure 2. MSI-X Mode vs. Legacy Mode |
360 | 365 | ||
361 | In Figure 2.0, a device operates by default in legacy mode. A | 366 | In Figure 2, a device operates by default in legacy mode. A |
362 | successful MSI-X request (using pci_enable_msix()) switches a | 367 | successful MSI-X request (using pci_enable_msix()) switches a |
363 | device's interrupt mode to MSI-X mode. A pre-assigned IOAPIC vector | 368 | device's interrupt mode to MSI-X mode. A pre-assigned IOAPIC vector |
364 | stored in dev->irq will be saved by the PCI subsystem; however, | 369 | stored in dev->irq will be saved by the PCI subsystem; however, |
365 | unlike MSI mode, the PCI subsystem will not replace dev->irq with | 370 | unlike MSI mode, the PCI subsystem will not replace dev->irq with |
366 | assigned MSI-X vector because the PCI subsystem already writes the 1:1 | 371 | assigned MSI-X vector because the PCI subsystem already writes the 1:1 |
367 | vector-to-entry mapping into the field vector of each element | 372 | vector-to-entry mapping into the field 'vector' of each element |
368 | specified in second argument. | 373 | specified in second argument. |
369 | 374 | ||
370 | To return back to its default mode, a device driver should always call | 375 | To return back to its default mode, a device driver should always call |
371 | pci_disable_msix() to undo the effect of pci_enable_msix(). Note that | 376 | pci_disable_msix() to undo the effect of pci_enable_msix(). Note that |
372 | a device driver should always call free_irq() on all MSI-X vectors it | 377 | a device driver should always call free_irq() on all MSI-X vectors it |
373 | has done request_irq() on before calling pci_disable_msix(). Failure | 378 | has done request_irq() on before calling pci_disable_msix(). Failure |
374 | to do so results a BUG_ON() and a device will be left with MSI-X | 379 | to do so results in a BUG_ON() and a device will be left with MSI-X |
375 | enabled and leaks its vectors. Otherwise, the PCI subsystem switches a | 380 | enabled and leaks its vectors. Otherwise, the PCI subsystem switches a |
376 | device function's interrupt mode from MSI-X mode to legacy mode and | 381 | device function's interrupt mode from MSI-X mode to legacy mode and |
377 | marks all allocated MSI-X vectors as unused. | 382 | marks all allocated MSI-X vectors as unused. |
@@ -383,53 +388,56 @@ MSI/MSI-X requests from other drivers, these MSI-X vectors may be | |||
383 | re-assigned. | 388 | re-assigned. |
384 | 389 | ||
385 | For the case where the PCI subsystem re-assigned these MSI-X vectors | 390 | For the case where the PCI subsystem re-assigned these MSI-X vectors |
386 | to other driver, a request to switching back to MSI-X mode may result | 391 | to other drivers, a request to switch back to MSI-X mode may result |
387 | being assigned with another set of MSI-X vectors or a failure if no | 392 | being assigned with another set of MSI-X vectors or a failure if no |
388 | more vectors are available. | 393 | more vectors are available. |
389 | 394 | ||
390 | 5.4 Handling function implementng both MSI and MSI-X capabilities | 395 | 5.4 Handling function implementing both MSI and MSI-X capabilities |
391 | 396 | ||
392 | For the case where a function implements both MSI and MSI-X | 397 | For the case where a function implements both MSI and MSI-X |
393 | capabilities, the PCI subsystem enables a device to run either in MSI | 398 | capabilities, the PCI subsystem enables a device to run either in MSI |
394 | mode or MSI-X mode but not both. A device driver determines whether it | 399 | mode or MSI-X mode but not both. A device driver determines whether it |
395 | wants MSI or MSI-X enabled on its hardware device. Once a device | 400 | wants MSI or MSI-X enabled on its hardware device. Once a device |
396 | driver requests for MSI, for example, it is prohibited to request for | 401 | driver requests for MSI, for example, it is prohibited from requesting |
397 | MSI-X; in other words, a device driver is not permitted to ping-pong | 402 | MSI-X; in other words, a device driver is not permitted to ping-pong |
398 | between MSI mod MSI-X mode during a run-time. | 403 | between MSI mod MSI-X mode during a run-time. |
399 | 404 | ||
400 | 5.5 Hardware requirements for MSI/MSI-X support | 405 | 5.5 Hardware requirements for MSI/MSI-X support |
406 | |||
401 | MSI/MSI-X support requires support from both system hardware and | 407 | MSI/MSI-X support requires support from both system hardware and |
402 | individual hardware device functions. | 408 | individual hardware device functions. |
403 | 409 | ||
404 | 5.5.1 System hardware support | 410 | 5.5.1 System hardware support |
411 | |||
405 | Since the target of MSI address is the local APIC CPU, enabling | 412 | Since the target of MSI address is the local APIC CPU, enabling |
406 | MSI/MSI-X support in Linux kernel is dependent on whether existing | 413 | MSI/MSI-X support in the Linux kernel is dependent on whether existing |
407 | system hardware supports local APIC. Users should verify their | 414 | system hardware supports local APIC. Users should verify that their |
408 | system whether it runs when CONFIG_X86_LOCAL_APIC=y. | 415 | system supports local APIC operation by testing that it runs when |
416 | CONFIG_X86_LOCAL_APIC=y. | ||
409 | 417 | ||
410 | In SMP environment, CONFIG_X86_LOCAL_APIC is automatically set; | 418 | In SMP environment, CONFIG_X86_LOCAL_APIC is automatically set; |
411 | however, in UP environment, users must manually set | 419 | however, in UP environment, users must manually set |
412 | CONFIG_X86_LOCAL_APIC. Once CONFIG_X86_LOCAL_APIC=y, setting | 420 | CONFIG_X86_LOCAL_APIC. Once CONFIG_X86_LOCAL_APIC=y, setting |
413 | CONFIG_PCI_MSI enables the VECTOR based scheme and | 421 | CONFIG_PCI_MSI enables the VECTOR based scheme and the option for |
414 | the option for MSI-capable device drivers to selectively enable | 422 | MSI-capable device drivers to selectively enable MSI/MSI-X. |
415 | MSI/MSI-X. | ||
416 | 423 | ||
417 | Note that CONFIG_X86_IO_APIC setting is irrelevant because MSI/MSI-X | 424 | Note that CONFIG_X86_IO_APIC setting is irrelevant because MSI/MSI-X |
418 | vector is allocated new during runtime and MSI/MSI-X support does not | 425 | vector is allocated new during runtime and MSI/MSI-X support does not |
419 | depend on BIOS support. This key independency enables MSI/MSI-X | 426 | depend on BIOS support. This key independency enables MSI/MSI-X |
420 | support on future IOxAPIC free platform. | 427 | support on future IOxAPIC free platforms. |
421 | 428 | ||
422 | 5.5.2 Device hardware support | 429 | 5.5.2 Device hardware support |
430 | |||
423 | The hardware device function supports MSI by indicating the | 431 | The hardware device function supports MSI by indicating the |
424 | MSI/MSI-X capability structure on its PCI capability list. By | 432 | MSI/MSI-X capability structure on its PCI capability list. By |
425 | default, this capability structure will not be initialized by | 433 | default, this capability structure will not be initialized by |
426 | the kernel to enable MSI during the system boot. In other words, | 434 | the kernel to enable MSI during the system boot. In other words, |
427 | the device function is running on its default pin assertion mode. | 435 | the device function is running on its default pin assertion mode. |
428 | Note that in many cases the hardware supporting MSI have bugs, | 436 | Note that in many cases the hardware supporting MSI have bugs, |
429 | which may result in system hang. The software driver of specific | 437 | which may result in system hangs. The software driver of specific |
430 | MSI-capable hardware is responsible for whether calling | 438 | MSI-capable hardware is responsible for deciding whether to call |
431 | pci_enable_msi or not. A return of zero indicates the kernel | 439 | pci_enable_msi or not. A return of zero indicates the kernel |
432 | successfully initializes the MSI/MSI-X capability structure of the | 440 | successfully initialized the MSI/MSI-X capability structure of the |
433 | device function. The device function is now running on MSI/MSI-X mode. | 441 | device function. The device function is now running on MSI/MSI-X mode. |
434 | 442 | ||
435 | 5.6 How to tell whether MSI/MSI-X is enabled on device function | 443 | 5.6 How to tell whether MSI/MSI-X is enabled on device function |
@@ -439,10 +447,10 @@ pci_enable_msi()/pci_enable_msix() indicates to a device driver that | |||
439 | its device function is initialized successfully and ready to run in | 447 | its device function is initialized successfully and ready to run in |
440 | MSI/MSI-X mode. | 448 | MSI/MSI-X mode. |
441 | 449 | ||
442 | At the user level, users can use command 'cat /proc/interrupts' | 450 | At the user level, users can use the command 'cat /proc/interrupts' |
443 | to display the vector allocated for a device and its interrupt | 451 | to display the vectors allocated for devices and their interrupt |
444 | MSI/MSI-X mode ("PCI MSI"/"PCI MSIX"). Below shows below MSI mode is | 452 | MSI/MSI-X modes ("PCI-MSI"/"PCI-MSI-X"). Below shows MSI mode is |
445 | enabled on a SCSI Adaptec 39320D Ultra320. | 453 | enabled on a SCSI Adaptec 39320D Ultra320 controller. |
446 | 454 | ||
447 | CPU0 CPU1 | 455 | CPU0 CPU1 |
448 | 0: 324639 0 IO-APIC-edge timer | 456 | 0: 324639 0 IO-APIC-edge timer |
@@ -453,8 +461,8 @@ enabled on a SCSI Adaptec 39320D Ultra320. | |||
453 | 15: 1 0 IO-APIC-edge ide1 | 461 | 15: 1 0 IO-APIC-edge ide1 |
454 | 169: 0 0 IO-APIC-level uhci-hcd | 462 | 169: 0 0 IO-APIC-level uhci-hcd |
455 | 185: 0 0 IO-APIC-level uhci-hcd | 463 | 185: 0 0 IO-APIC-level uhci-hcd |
456 | 193: 138 10 PCI MSI aic79xx | 464 | 193: 138 10 PCI-MSI aic79xx |
457 | 201: 30 0 PCI MSI aic79xx | 465 | 201: 30 0 PCI-MSI aic79xx |
458 | 225: 30 0 IO-APIC-level aic7xxx | 466 | 225: 30 0 IO-APIC-level aic7xxx |
459 | 233: 30 0 IO-APIC-level aic7xxx | 467 | 233: 30 0 IO-APIC-level aic7xxx |
460 | NMI: 0 0 | 468 | NMI: 0 0 |
@@ -490,8 +498,8 @@ target address set as 0xfeexxxxx, as conformed to PCI | |||
490 | specification 2.3 or latest, then it should work. | 498 | specification 2.3 or latest, then it should work. |
491 | 499 | ||
492 | Q4. From the driver point of view, if the MSI is lost because | 500 | Q4. From the driver point of view, if the MSI is lost because |
493 | of the errors occur during inbound memory write, then it may | 501 | of errors occurring during inbound memory write, then it may |
494 | wait for ever. Is there a mechanism for it to recover? | 502 | wait forever. Is there a mechanism for it to recover? |
495 | 503 | ||
496 | A4. Since the target of the transaction is an inbound memory | 504 | A4. Since the target of the transaction is an inbound memory |
497 | write, all transaction termination conditions (Retry, | 505 | write, all transaction termination conditions (Retry, |