diff options
author | Randy Dunlap <randy.dunlap@oracle.com> | 2008-11-13 16:33:24 -0500 |
---|---|---|
committer | Randy Dunlap <randy.dunlap@oracle.com> | 2008-11-14 12:28:53 -0500 |
commit | 31c00fc15ebd35c1647775dbfc167a15d46657fd (patch) | |
tree | 6d8ff2a6607c94a791ccc56fd8eb625e4fdcc01a /Documentation/MSI-HOWTO.txt | |
parent | 3edac25f2e8ac8c2a84904c140e1aeb434e73e75 (diff) |
Create/use more directory structure in the Documentation/ tree.
Create Documentation/blockdev/ sub-directory and populate it.
Populate the Documentation/serial/ sub-directory.
Move MSI-HOWTO.txt to Documentation/PCI/.
Move ioctl-number.txt to Documentation/ioctl/.
Update all relevant 00-INDEX files.
Update all relevant Kconfig files and source files.
Signed-off-by: Randy Dunlap <randy.dunlap@oracle.com>
Diffstat (limited to 'Documentation/MSI-HOWTO.txt')
-rw-r--r-- | Documentation/MSI-HOWTO.txt | 509 |
1 files changed, 0 insertions, 509 deletions
diff --git a/Documentation/MSI-HOWTO.txt b/Documentation/MSI-HOWTO.txt deleted file mode 100644 index 256defd7e174..000000000000 --- a/Documentation/MSI-HOWTO.txt +++ /dev/null | |||
@@ -1,509 +0,0 @@ | |||
1 | The MSI Driver Guide HOWTO | ||
2 | Tom L Nguyen tom.l.nguyen@intel.com | ||
3 | 10/03/2003 | ||
4 | Revised Feb 12, 2004 by Martine Silbermann | ||
5 | email: Martine.Silbermann@hp.com | ||
6 | Revised Jun 25, 2004 by Tom L Nguyen | ||
7 | |||
8 | 1. About this guide | ||
9 | |||
10 | This guide describes the basics of Message Signaled Interrupts (MSI), | ||
11 | the advantages of using MSI over traditional interrupt mechanisms, | ||
12 | and how to enable your driver to use MSI or MSI-X. Also included is | ||
13 | a Frequently Asked Questions (FAQ) section. | ||
14 | |||
15 | 1.1 Terminology | ||
16 | |||
17 | PCI devices can be single-function or multi-function. In either case, | ||
18 | when this text talks about enabling or disabling MSI on a "device | ||
19 | function," it is referring to one specific PCI device and function and | ||
20 | not to all functions on a PCI device (unless the PCI device has only | ||
21 | one function). | ||
22 | |||
23 | 2. Copyright 2003 Intel Corporation | ||
24 | |||
25 | 3. What is MSI/MSI-X? | ||
26 | |||
27 | Message Signaled Interrupt (MSI), as described in the PCI Local Bus | ||
28 | Specification Revision 2.3 or later, is an optional feature, and a | ||
29 | required feature for PCI Express devices. MSI enables a device function | ||
30 | to request service by sending an Inbound Memory Write on its PCI bus to | ||
31 | the FSB as a Message Signal Interrupt transaction. Because MSI is | ||
32 | generated in the form of a Memory Write, all transaction conditions, | ||
33 | such as a Retry, Master-Abort, Target-Abort or normal completion, are | ||
34 | supported. | ||
35 | |||
36 | A PCI device that supports MSI must also support pin IRQ assertion | ||
37 | interrupt mechanism to provide backward compatibility for systems that | ||
38 | do not support MSI. In systems which support MSI, the bus driver is | ||
39 | responsible for initializing the message address and message data of | ||
40 | the device function's MSI/MSI-X capability structure during device | ||
41 | initial configuration. | ||
42 | |||
43 | An MSI capable device function indicates MSI support by implementing | ||
44 | the MSI/MSI-X capability structure in its PCI capability list. The | ||
45 | device function may implement both the MSI capability structure and | ||
46 | the MSI-X capability structure; however, the bus driver should not | ||
47 | enable both. | ||
48 | |||
49 | The MSI capability structure contains Message Control register, | ||
50 | Message Address register and Message Data register. These registers | ||
51 | provide the bus driver control over MSI. The Message Control register | ||
52 | indicates the MSI capability supported by the device. The Message | ||
53 | Address register specifies the target address and the Message Data | ||
54 | register specifies the characteristics of the message. To request | ||
55 | service, the device function writes the content of the Message Data | ||
56 | register to the target address. The device and its software driver | ||
57 | are prohibited from writing to these registers. | ||
58 | |||
59 | The MSI-X capability structure is an optional extension to MSI. It | ||
60 | uses an independent and separate capability structure. There are | ||
61 | some key advantages to implementing the MSI-X capability structure | ||
62 | over the MSI capability structure as described below. | ||
63 | |||
64 | - Support a larger maximum number of vectors per function. | ||
65 | |||
66 | - Provide the ability for system software to configure | ||
67 | each vector with an independent message address and message | ||
68 | data, specified by a table that resides in Memory Space. | ||
69 | |||
70 | - MSI and MSI-X both support per-vector masking. Per-vector | ||
71 | masking is an optional extension of MSI but a required | ||
72 | feature for MSI-X. Per-vector masking provides the kernel the | ||
73 | ability to mask/unmask a single MSI while running its | ||
74 | interrupt service routine. If per-vector masking is | ||
75 | not supported, then the device driver should provide the | ||
76 | hardware/software synchronization to ensure that the device | ||
77 | generates MSI when the driver wants it to do so. | ||
78 | |||
79 | 4. Why use MSI? | ||
80 | |||
81 | As a benefit to the simplification of board design, MSI allows board | ||
82 | designers to remove out-of-band interrupt routing. MSI is another | ||
83 | step towards a legacy-free environment. | ||
84 | |||
85 | Due to increasing pressure on chipset and processor packages to | ||
86 | reduce pin count, the need for interrupt pins is expected to | ||
87 | diminish over time. Devices, due to pin constraints, may implement | ||
88 | messages to increase performance. | ||
89 | |||
90 | PCI Express endpoints uses INTx emulation (in-band messages) instead | ||
91 | of IRQ pin assertion. Using INTx emulation requires interrupt | ||
92 | sharing among devices connected to the same node (PCI bridge) while | ||
93 | MSI is unique (non-shared) and does not require BIOS configuration | ||
94 | support. As a result, the PCI Express technology requires MSI | ||
95 | support for better interrupt performance. | ||
96 | |||
97 | Using MSI enables the device functions to support two or more | ||
98 | vectors, which can be configured to target different CPUs to | ||
99 | increase scalability. | ||
100 | |||
101 | 5. Configuring a driver to use MSI/MSI-X | ||
102 | |||
103 | By default, the kernel will not enable MSI/MSI-X on all devices that | ||
104 | support this capability. The CONFIG_PCI_MSI kernel option | ||
105 | must be selected to enable MSI/MSI-X support. | ||
106 | |||
107 | 5.1 Including MSI/MSI-X support into the kernel | ||
108 | |||
109 | To allow MSI/MSI-X capable device drivers to selectively enable | ||
110 | MSI/MSI-X (using pci_enable_msi()/pci_enable_msix() as described | ||
111 | below), the VECTOR based scheme needs to be enabled by setting | ||
112 | CONFIG_PCI_MSI during kernel config. | ||
113 | |||
114 | Since the target of the inbound message is the local APIC, providing | ||
115 | CONFIG_X86_LOCAL_APIC must be enabled as well as CONFIG_PCI_MSI. | ||
116 | |||
117 | 5.2 Configuring for MSI support | ||
118 | |||
119 | Due to the non-contiguous fashion in vector assignment of the | ||
120 | existing Linux kernel, this version does not support multiple | ||
121 | messages regardless of a device function is capable of supporting | ||
122 | more than one vector. To enable MSI on a device function's MSI | ||
123 | capability structure requires a device driver to call the function | ||
124 | pci_enable_msi() explicitly. | ||
125 | |||
126 | 5.2.1 API pci_enable_msi | ||
127 | |||
128 | int pci_enable_msi(struct pci_dev *dev) | ||
129 | |||
130 | With this new API, a device driver that wants to have MSI | ||
131 | enabled on its device function must call this API to enable MSI. | ||
132 | A successful call will initialize the MSI capability structure | ||
133 | with ONE vector, regardless of whether a device function is | ||
134 | capable of supporting multiple messages. This vector replaces the | ||
135 | pre-assigned dev->irq with a new MSI vector. To avoid a conflict | ||
136 | of the new assigned vector with existing pre-assigned vector requires | ||
137 | a device driver to call this API before calling request_irq(). | ||
138 | |||
139 | 5.2.2 API pci_disable_msi | ||
140 | |||
141 | void pci_disable_msi(struct pci_dev *dev) | ||
142 | |||
143 | This API should always be used to undo the effect of pci_enable_msi() | ||
144 | when a device driver is unloading. This API restores dev->irq with | ||
145 | the pre-assigned IOAPIC vector and switches a device's interrupt | ||
146 | mode to PCI pin-irq assertion/INTx emulation mode. | ||
147 | |||
148 | Note that a device driver should always call free_irq() on the MSI vector | ||
149 | that it has done request_irq() on before calling this API. Failure to do | ||
150 | so results in a BUG_ON() and a device will be left with MSI enabled and | ||
151 | leaks its vector. | ||
152 | |||
153 | 5.2.3 MSI mode vs. legacy mode diagram | ||
154 | |||
155 | The below diagram shows the events which switch the interrupt | ||
156 | mode on the MSI-capable device function between MSI mode and | ||
157 | PIN-IRQ assertion mode. | ||
158 | |||
159 | ------------ pci_enable_msi ------------------------ | ||
160 | | | <=============== | | | ||
161 | | MSI MODE | | PIN-IRQ ASSERTION MODE | | ||
162 | | | ===============> | | | ||
163 | ------------ pci_disable_msi ------------------------ | ||
164 | |||
165 | |||
166 | Figure 1. MSI Mode vs. Legacy Mode | ||
167 | |||
168 | In Figure 1, a device operates by default in legacy mode. Legacy | ||
169 | in this context means PCI pin-irq assertion or PCI-Express INTx | ||
170 | emulation. A successful MSI request (using pci_enable_msi()) switches | ||
171 | a device's interrupt mode to MSI mode. A pre-assigned IOAPIC vector | ||
172 | stored in dev->irq will be saved by the PCI subsystem and a new | ||
173 | assigned MSI vector will replace dev->irq. | ||
174 | |||
175 | To return back to its default mode, a device driver should always call | ||
176 | pci_disable_msi() to undo the effect of pci_enable_msi(). Note that a | ||
177 | device driver should always call free_irq() on the MSI vector it has | ||
178 | done request_irq() on before calling pci_disable_msi(). Failure to do | ||
179 | so results in a BUG_ON() and a device will be left with MSI enabled and | ||
180 | leaks its vector. Otherwise, the PCI subsystem restores a device's | ||
181 | dev->irq with a pre-assigned IOAPIC vector and marks the released | ||
182 | MSI vector as unused. | ||
183 | |||
184 | Once being marked as unused, there is no guarantee that the PCI | ||
185 | subsystem will reserve this MSI vector for a device. Depending on | ||
186 | the availability of current PCI vector resources and the number of | ||
187 | MSI/MSI-X requests from other drivers, this MSI may be re-assigned. | ||
188 | |||
189 | For the case where the PCI subsystem re-assigns this MSI vector to | ||
190 | another driver, a request to switch back to MSI mode may result | ||
191 | in being assigned a different MSI vector or a failure if no more | ||
192 | vectors are available. | ||
193 | |||
194 | 5.3 Configuring for MSI-X support | ||
195 | |||
196 | Due to the ability of the system software to configure each vector of | ||
197 | the MSI-X capability structure with an independent message address | ||
198 | and message data, the non-contiguous fashion in vector assignment of | ||
199 | the existing Linux kernel has no impact on supporting multiple | ||
200 | messages on an MSI-X capable device functions. To enable MSI-X on | ||
201 | a device function's MSI-X capability structure requires its device | ||
202 | driver to call the function pci_enable_msix() explicitly. | ||
203 | |||
204 | The function pci_enable_msix(), once invoked, enables either | ||
205 | all or nothing, depending on the current availability of PCI vector | ||
206 | resources. If the PCI vector resources are available for the number | ||
207 | of vectors requested by a device driver, this function will configure | ||
208 | the MSI-X table of the MSI-X capability structure of a device with | ||
209 | requested messages. To emphasize this reason, for example, a device | ||
210 | may be capable for supporting the maximum of 32 vectors while its | ||
211 | software driver usually may request 4 vectors. It is recommended | ||
212 | that the device driver should call this function once during the | ||
213 | initialization phase of the device driver. | ||
214 | |||
215 | Unlike the function pci_enable_msi(), the function pci_enable_msix() | ||
216 | does not replace the pre-assigned IOAPIC dev->irq with a new MSI | ||
217 | vector because the PCI subsystem writes the 1:1 vector-to-entry mapping | ||
218 | into the field vector of each element contained in a second argument. | ||
219 | Note that the pre-assigned IOAPIC dev->irq is valid only if the device | ||
220 | operates in PIN-IRQ assertion mode. In MSI-X mode, any attempt at | ||
221 | using dev->irq by the device driver to request for interrupt service | ||
222 | may result in unpredictable behavior. | ||
223 | |||
224 | For each MSI-X vector granted, a device driver is responsible for calling | ||
225 | other functions like request_irq(), enable_irq(), etc. to enable | ||
226 | this vector with its corresponding interrupt service handler. It is | ||
227 | a device driver's choice to assign all vectors with the same | ||
228 | interrupt service handler or each vector with a unique interrupt | ||
229 | service handler. | ||
230 | |||
231 | 5.3.1 Handling MMIO address space of MSI-X Table | ||
232 | |||
233 | The PCI 3.0 specification has implementation notes that MMIO address | ||
234 | space for a device's MSI-X structure should be isolated so that the | ||
235 | software system can set different pages for controlling accesses to the | ||
236 | MSI-X structure. The implementation of MSI support requires the PCI | ||
237 | subsystem, not a device driver, to maintain full control of the MSI-X | ||
238 | table/MSI-X PBA (Pending Bit Array) and MMIO address space of the MSI-X | ||
239 | table/MSI-X PBA. A device driver should not access the MMIO address | ||
240 | space of the MSI-X table/MSI-X PBA. | ||
241 | |||
242 | 5.3.2 API pci_enable_msix | ||
243 | |||
244 | int pci_enable_msix(struct pci_dev *dev, struct msix_entry *entries, int nvec) | ||
245 | |||
246 | This API enables a device driver to request the PCI subsystem | ||
247 | to enable MSI-X messages on its hardware device. Depending on | ||
248 | the availability of PCI vectors resources, the PCI subsystem enables | ||
249 | either all or none of the requested vectors. | ||
250 | |||
251 | Argument 'dev' points to the device (pci_dev) structure. | ||
252 | |||
253 | Argument 'entries' is a pointer to an array of msix_entry structs. | ||
254 | The number of entries is indicated in argument 'nvec'. | ||
255 | struct msix_entry is defined in /driver/pci/msi.h: | ||
256 | |||
257 | struct msix_entry { | ||
258 | u16 vector; /* kernel uses to write alloc vector */ | ||
259 | u16 entry; /* driver uses to specify entry */ | ||
260 | }; | ||
261 | |||
262 | A device driver is responsible for initializing the field 'entry' of | ||
263 | each element with a unique entry supported by MSI-X table. Otherwise, | ||
264 | -EINVAL will be returned as a result. A successful return of zero | ||
265 | indicates the PCI subsystem completed initializing each of the requested | ||
266 | entries of the MSI-X table with message address and message data. | ||
267 | Last but not least, the PCI subsystem will write the 1:1 | ||
268 | vector-to-entry mapping into the field 'vector' of each element. A | ||
269 | device driver is responsible for keeping track of allocated MSI-X | ||
270 | vectors in its internal data structure. | ||
271 | |||
272 | A return of zero indicates that the number of MSI-X vectors was | ||
273 | successfully allocated. A return of greater than zero indicates | ||
274 | MSI-X vector shortage. Or a return of less than zero indicates | ||
275 | a failure. This failure may be a result of duplicate entries | ||
276 | specified in second argument, or a result of no available vector, | ||
277 | or a result of failing to initialize MSI-X table entries. | ||
278 | |||
279 | 5.3.3 API pci_disable_msix | ||
280 | |||
281 | void pci_disable_msix(struct pci_dev *dev) | ||
282 | |||
283 | This API should always be used to undo the effect of pci_enable_msix() | ||
284 | when a device driver is unloading. Note that a device driver should | ||
285 | always call free_irq() on all MSI-X vectors it has done request_irq() | ||
286 | on before calling this API. Failure to do so results in a BUG_ON() and | ||
287 | a device will be left with MSI-X enabled and leaks its vectors. | ||
288 | |||
289 | 5.3.4 MSI-X mode vs. legacy mode diagram | ||
290 | |||
291 | The below diagram shows the events which switch the interrupt | ||
292 | mode on the MSI-X capable device function between MSI-X mode and | ||
293 | PIN-IRQ assertion mode (legacy). | ||
294 | |||
295 | ------------ pci_enable_msix(,,n) ------------------------ | ||
296 | | | <=============== | | | ||
297 | | MSI-X MODE | | PIN-IRQ ASSERTION MODE | | ||
298 | | | ===============> | | | ||
299 | ------------ pci_disable_msix ------------------------ | ||
300 | |||
301 | Figure 2. MSI-X Mode vs. Legacy Mode | ||
302 | |||
303 | In Figure 2, a device operates by default in legacy mode. A | ||
304 | successful MSI-X request (using pci_enable_msix()) switches a | ||
305 | device's interrupt mode to MSI-X mode. A pre-assigned IOAPIC vector | ||
306 | stored in dev->irq will be saved by the PCI subsystem; however, | ||
307 | unlike MSI mode, the PCI subsystem will not replace dev->irq with | ||
308 | assigned MSI-X vector because the PCI subsystem already writes the 1:1 | ||
309 | vector-to-entry mapping into the field 'vector' of each element | ||
310 | specified in second argument. | ||
311 | |||
312 | To return back to its default mode, a device driver should always call | ||
313 | pci_disable_msix() to undo the effect of pci_enable_msix(). Note that | ||
314 | a device driver should always call free_irq() on all MSI-X vectors it | ||
315 | has done request_irq() on before calling pci_disable_msix(). Failure | ||
316 | to do so results in a BUG_ON() and a device will be left with MSI-X | ||
317 | enabled and leaks its vectors. Otherwise, the PCI subsystem switches a | ||
318 | device function's interrupt mode from MSI-X mode to legacy mode and | ||
319 | marks all allocated MSI-X vectors as unused. | ||
320 | |||
321 | Once being marked as unused, there is no guarantee that the PCI | ||
322 | subsystem will reserve these MSI-X vectors for a device. Depending on | ||
323 | the availability of current PCI vector resources and the number of | ||
324 | MSI/MSI-X requests from other drivers, these MSI-X vectors may be | ||
325 | re-assigned. | ||
326 | |||
327 | For the case where the PCI subsystem re-assigned these MSI-X vectors | ||
328 | to other drivers, a request to switch back to MSI-X mode may result | ||
329 | being assigned with another set of MSI-X vectors or a failure if no | ||
330 | more vectors are available. | ||
331 | |||
332 | 5.4 Handling function implementing both MSI and MSI-X capabilities | ||
333 | |||
334 | For the case where a function implements both MSI and MSI-X | ||
335 | capabilities, the PCI subsystem enables a device to run either in MSI | ||
336 | mode or MSI-X mode but not both. A device driver determines whether it | ||
337 | wants MSI or MSI-X enabled on its hardware device. Once a device | ||
338 | driver requests for MSI, for example, it is prohibited from requesting | ||
339 | MSI-X; in other words, a device driver is not permitted to ping-pong | ||
340 | between MSI mod MSI-X mode during a run-time. | ||
341 | |||
342 | 5.5 Hardware requirements for MSI/MSI-X support | ||
343 | |||
344 | MSI/MSI-X support requires support from both system hardware and | ||
345 | individual hardware device functions. | ||
346 | |||
347 | 5.5.1 Required x86 hardware support | ||
348 | |||
349 | Since the target of MSI address is the local APIC CPU, enabling | ||
350 | MSI/MSI-X support in the Linux kernel is dependent on whether existing | ||
351 | system hardware supports local APIC. Users should verify that their | ||
352 | system supports local APIC operation by testing that it runs when | ||
353 | CONFIG_X86_LOCAL_APIC=y. | ||
354 | |||
355 | In SMP environment, CONFIG_X86_LOCAL_APIC is automatically set; | ||
356 | however, in UP environment, users must manually set | ||
357 | CONFIG_X86_LOCAL_APIC. Once CONFIG_X86_LOCAL_APIC=y, setting | ||
358 | CONFIG_PCI_MSI enables the VECTOR based scheme and the option for | ||
359 | MSI-capable device drivers to selectively enable MSI/MSI-X. | ||
360 | |||
361 | Note that CONFIG_X86_IO_APIC setting is irrelevant because MSI/MSI-X | ||
362 | vector is allocated new during runtime and MSI/MSI-X support does not | ||
363 | depend on BIOS support. This key independency enables MSI/MSI-X | ||
364 | support on future IOxAPIC free platforms. | ||
365 | |||
366 | 5.5.2 Device hardware support | ||
367 | |||
368 | The hardware device function supports MSI by indicating the | ||
369 | MSI/MSI-X capability structure on its PCI capability list. By | ||
370 | default, this capability structure will not be initialized by | ||
371 | the kernel to enable MSI during the system boot. In other words, | ||
372 | the device function is running on its default pin assertion mode. | ||
373 | Note that in many cases the hardware supporting MSI have bugs, | ||
374 | which may result in system hangs. The software driver of specific | ||
375 | MSI-capable hardware is responsible for deciding whether to call | ||
376 | pci_enable_msi or not. A return of zero indicates the kernel | ||
377 | successfully initialized the MSI/MSI-X capability structure of the | ||
378 | device function. The device function is now running on MSI/MSI-X mode. | ||
379 | |||
380 | 5.6 How to tell whether MSI/MSI-X is enabled on device function | ||
381 | |||
382 | At the driver level, a return of zero from the function call of | ||
383 | pci_enable_msi()/pci_enable_msix() indicates to a device driver that | ||
384 | its device function is initialized successfully and ready to run in | ||
385 | MSI/MSI-X mode. | ||
386 | |||
387 | At the user level, users can use the command 'cat /proc/interrupts' | ||
388 | to display the vectors allocated for devices and their interrupt | ||
389 | MSI/MSI-X modes ("PCI-MSI"/"PCI-MSI-X"). Below shows MSI mode is | ||
390 | enabled on a SCSI Adaptec 39320D Ultra320 controller. | ||
391 | |||
392 | CPU0 CPU1 | ||
393 | 0: 324639 0 IO-APIC-edge timer | ||
394 | 1: 1186 0 IO-APIC-edge i8042 | ||
395 | 2: 0 0 XT-PIC cascade | ||
396 | 12: 2797 0 IO-APIC-edge i8042 | ||
397 | 14: 6543 0 IO-APIC-edge ide0 | ||
398 | 15: 1 0 IO-APIC-edge ide1 | ||
399 | 169: 0 0 IO-APIC-level uhci-hcd | ||
400 | 185: 0 0 IO-APIC-level uhci-hcd | ||
401 | 193: 138 10 PCI-MSI aic79xx | ||
402 | 201: 30 0 PCI-MSI aic79xx | ||
403 | 225: 30 0 IO-APIC-level aic7xxx | ||
404 | 233: 30 0 IO-APIC-level aic7xxx | ||
405 | NMI: 0 0 | ||
406 | LOC: 324553 325068 | ||
407 | ERR: 0 | ||
408 | MIS: 0 | ||
409 | |||
410 | 6. MSI quirks | ||
411 | |||
412 | Several PCI chipsets or devices are known to not support MSI. | ||
413 | The PCI stack provides 3 possible levels of MSI disabling: | ||
414 | * on a single device | ||
415 | * on all devices behind a specific bridge | ||
416 | * globally | ||
417 | |||
418 | 6.1. Disabling MSI on a single device | ||
419 | |||
420 | Under some circumstances it might be required to disable MSI on a | ||
421 | single device. This may be achieved by either not calling pci_enable_msi() | ||
422 | or all, or setting the pci_dev->no_msi flag before (most of the time | ||
423 | in a quirk). | ||
424 | |||
425 | 6.2. Disabling MSI below a bridge | ||
426 | |||
427 | The vast majority of MSI quirks are required by PCI bridges not | ||
428 | being able to route MSI between busses. In this case, MSI have to be | ||
429 | disabled on all devices behind this bridge. It is achieves by setting | ||
430 | the PCI_BUS_FLAGS_NO_MSI flag in the pci_bus->bus_flags of the bridge | ||
431 | subordinate bus. There is no need to set the same flag on bridges that | ||
432 | are below the broken bridge. When pci_enable_msi() is called to enable | ||
433 | MSI on a device, pci_msi_supported() takes care of checking the NO_MSI | ||
434 | flag in all parent busses of the device. | ||
435 | |||
436 | Some bridges actually support dynamic MSI support enabling/disabling | ||
437 | by changing some bits in their PCI configuration space (especially | ||
438 | the Hypertransport chipsets such as the nVidia nForce and Serverworks | ||
439 | HT2000). It may then be required to update the NO_MSI flag on the | ||
440 | corresponding devices in the sysfs hierarchy. To enable MSI support | ||
441 | on device "0000:00:0e", do: | ||
442 | |||
443 | echo 1 > /sys/bus/pci/devices/0000:00:0e/msi_bus | ||
444 | |||
445 | To disable MSI support, echo 0 instead of 1. Note that it should be | ||
446 | used with caution since changing this value might break interrupts. | ||
447 | |||
448 | 6.3. Disabling MSI globally | ||
449 | |||
450 | Some extreme cases may require to disable MSI globally on the system. | ||
451 | For now, the only known case is a Serverworks PCI-X chipsets (MSI are | ||
452 | not supported on several busses that are not all connected to the | ||
453 | chipset in the Linux PCI hierarchy). In the vast majority of other | ||
454 | cases, disabling only behind a specific bridge is enough. | ||
455 | |||
456 | For debugging purpose, the user may also pass pci=nomsi on the kernel | ||
457 | command-line to explicitly disable MSI globally. But, once the appro- | ||
458 | priate quirks are added to the kernel, this option should not be | ||
459 | required anymore. | ||
460 | |||
461 | 6.4. Finding why MSI cannot be enabled on a device | ||
462 | |||
463 | Assuming that MSI are not enabled on a device, you should look at | ||
464 | dmesg to find messages that quirks may output when disabling MSI | ||
465 | on some devices, some bridges or even globally. | ||
466 | Then, lspci -t gives the list of bridges above a device. Reading | ||
467 | /sys/bus/pci/devices/0000:00:0e/msi_bus will tell you whether MSI | ||
468 | are enabled (1) or disabled (0). In 0 is found in a single bridge | ||
469 | msi_bus file above the device, MSI cannot be enabled. | ||
470 | |||
471 | 7. FAQ | ||
472 | |||
473 | Q1. Are there any limitations on using the MSI? | ||
474 | |||
475 | A1. If the PCI device supports MSI and conforms to the | ||
476 | specification and the platform supports the APIC local bus, | ||
477 | then using MSI should work. | ||
478 | |||
479 | Q2. Will it work on all the Pentium processors (P3, P4, Xeon, | ||
480 | AMD processors)? In P3 IPI's are transmitted on the APIC local | ||
481 | bus and in P4 and Xeon they are transmitted on the system | ||
482 | bus. Are there any implications with this? | ||
483 | |||
484 | A2. MSI support enables a PCI device sending an inbound | ||
485 | memory write (0xfeexxxxx as target address) on its PCI bus | ||
486 | directly to the FSB. Since the message address has a | ||
487 | redirection hint bit cleared, it should work. | ||
488 | |||
489 | Q3. The target address 0xfeexxxxx will be translated by the | ||
490 | Host Bridge into an interrupt message. Are there any | ||
491 | limitations on the chipsets such as Intel 8xx, Intel e7xxx, | ||
492 | or VIA? | ||
493 | |||
494 | A3. If these chipsets support an inbound memory write with | ||
495 | target address set as 0xfeexxxxx, as conformed to PCI | ||
496 | specification 2.3 or latest, then it should work. | ||
497 | |||
498 | Q4. From the driver point of view, if the MSI is lost because | ||
499 | of errors occurring during inbound memory write, then it may | ||
500 | wait forever. Is there a mechanism for it to recover? | ||
501 | |||
502 | A4. Since the target of the transaction is an inbound memory | ||
503 | write, all transaction termination conditions (Retry, | ||
504 | Master-Abort, Target-Abort, or normal completion) are | ||
505 | supported. A device sending an MSI must abide by all the PCI | ||
506 | rules and conditions regarding that inbound memory write. So, | ||
507 | if a retry is signaled it must retry, etc... We believe that | ||
508 | the recommendation for Abort is also a retry (refer to PCI | ||
509 | specification 2.3 or latest). | ||