diff options
Diffstat (limited to 'Documentation/PCI/PCI-DMA-mapping.txt')
| -rw-r--r-- | Documentation/PCI/PCI-DMA-mapping.txt | 766 |
1 files changed, 766 insertions, 0 deletions
diff --git a/Documentation/PCI/PCI-DMA-mapping.txt b/Documentation/PCI/PCI-DMA-mapping.txt new file mode 100644 index 000000000000..ecad88d9fe59 --- /dev/null +++ b/Documentation/PCI/PCI-DMA-mapping.txt | |||
| @@ -0,0 +1,766 @@ | |||
| 1 | Dynamic DMA mapping | ||
| 2 | =================== | ||
| 3 | |||
| 4 | David S. Miller <davem@redhat.com> | ||
| 5 | Richard Henderson <rth@cygnus.com> | ||
| 6 | Jakub Jelinek <jakub@redhat.com> | ||
| 7 | |||
| 8 | This document describes the DMA mapping system in terms of the pci_ | ||
| 9 | API. For a similar API that works for generic devices, see | ||
| 10 | DMA-API.txt. | ||
| 11 | |||
| 12 | Most of the 64bit platforms have special hardware that translates bus | ||
| 13 | addresses (DMA addresses) into physical addresses. This is similar to | ||
| 14 | how page tables and/or a TLB translates virtual addresses to physical | ||
| 15 | addresses on a CPU. This is needed so that e.g. PCI devices can | ||
| 16 | access with a Single Address Cycle (32bit DMA address) any page in the | ||
| 17 | 64bit physical address space. Previously in Linux those 64bit | ||
| 18 | platforms had to set artificial limits on the maximum RAM size in the | ||
| 19 | system, so that the virt_to_bus() static scheme works (the DMA address | ||
| 20 | translation tables were simply filled on bootup to map each bus | ||
| 21 | address to the physical page __pa(bus_to_virt())). | ||
| 22 | |||
| 23 | So that Linux can use the dynamic DMA mapping, it needs some help from the | ||
| 24 | drivers, namely it has to take into account that DMA addresses should be | ||
| 25 | mapped only for the time they are actually used and unmapped after the DMA | ||
| 26 | transfer. | ||
| 27 | |||
| 28 | The following API will work of course even on platforms where no such | ||
| 29 | hardware exists, see e.g. arch/x86/include/asm/pci.h for how it is implemented on | ||
| 30 | top of the virt_to_bus interface. | ||
| 31 | |||
| 32 | First of all, you should make sure | ||
| 33 | |||
| 34 | #include <linux/pci.h> | ||
| 35 | |||
| 36 | is in your driver. This file will obtain for you the definition of the | ||
| 37 | dma_addr_t (which can hold any valid DMA address for the platform) | ||
| 38 | type which should be used everywhere you hold a DMA (bus) address | ||
| 39 | returned from the DMA mapping functions. | ||
| 40 | |||
| 41 | What memory is DMA'able? | ||
| 42 | |||
| 43 | The first piece of information you must know is what kernel memory can | ||
| 44 | be used with the DMA mapping facilities. There has been an unwritten | ||
| 45 | set of rules regarding this, and this text is an attempt to finally | ||
| 46 | write them down. | ||
| 47 | |||
| 48 | If you acquired your memory via the page allocator | ||
| 49 | (i.e. __get_free_page*()) or the generic memory allocators | ||
| 50 | (i.e. kmalloc() or kmem_cache_alloc()) then you may DMA to/from | ||
| 51 | that memory using the addresses returned from those routines. | ||
| 52 | |||
| 53 | This means specifically that you may _not_ use the memory/addresses | ||
| 54 | returned from vmalloc() for DMA. It is possible to DMA to the | ||
| 55 | _underlying_ memory mapped into a vmalloc() area, but this requires | ||
| 56 | walking page tables to get the physical addresses, and then | ||
| 57 | translating each of those pages back to a kernel address using | ||
| 58 | something like __va(). [ EDIT: Update this when we integrate | ||
| 59 | Gerd Knorr's generic code which does this. ] | ||
| 60 | |||
| 61 | This rule also means that you may use neither kernel image addresses | ||
| 62 | (items in data/text/bss segments), nor module image addresses, nor | ||
| 63 | stack addresses for DMA. These could all be mapped somewhere entirely | ||
| 64 | different than the rest of physical memory. Even if those classes of | ||
| 65 | memory could physically work with DMA, you'd need to ensure the I/O | ||
| 66 | buffers were cacheline-aligned. Without that, you'd see cacheline | ||
| 67 | sharing problems (data corruption) on CPUs with DMA-incoherent caches. | ||
| 68 | (The CPU could write to one word, DMA would write to a different one | ||
| 69 | in the same cache line, and one of them could be overwritten.) | ||
| 70 | |||
| 71 | Also, this means that you cannot take the return of a kmap() | ||
| 72 | call and DMA to/from that. This is similar to vmalloc(). | ||
| 73 | |||
| 74 | What about block I/O and networking buffers? The block I/O and | ||
| 75 | networking subsystems make sure that the buffers they use are valid | ||
| 76 | for you to DMA from/to. | ||
| 77 | |||
| 78 | DMA addressing limitations | ||
| 79 | |||
| 80 | Does your device have any DMA addressing limitations? For example, is | ||
| 81 | your device only capable of driving the low order 24-bits of address | ||
| 82 | on the PCI bus for SAC DMA transfers? If so, you need to inform the | ||
| 83 | PCI layer of this fact. | ||
| 84 | |||
| 85 | By default, the kernel assumes that your device can address the full | ||
| 86 | 32-bits in a SAC cycle. For a 64-bit DAC capable device, this needs | ||
| 87 | to be increased. And for a device with limitations, as discussed in | ||
| 88 | the previous paragraph, it needs to be decreased. | ||
| 89 | |||
| 90 | pci_alloc_consistent() by default will return 32-bit DMA addresses. | ||
| 91 | PCI-X specification requires PCI-X devices to support 64-bit | ||
| 92 | addressing (DAC) for all transactions. And at least one platform (SGI | ||
| 93 | SN2) requires 64-bit consistent allocations to operate correctly when | ||
| 94 | the IO bus is in PCI-X mode. Therefore, like with pci_set_dma_mask(), | ||
| 95 | it's good practice to call pci_set_consistent_dma_mask() to set the | ||
| 96 | appropriate mask even if your device only supports 32-bit DMA | ||
| 97 | (default) and especially if it's a PCI-X device. | ||
| 98 | |||
| 99 | For correct operation, you must interrogate the PCI layer in your | ||
| 100 | device probe routine to see if the PCI controller on the machine can | ||
| 101 | properly support the DMA addressing limitation your device has. It is | ||
| 102 | good style to do this even if your device holds the default setting, | ||
| 103 | because this shows that you did think about these issues wrt. your | ||
| 104 | device. | ||
| 105 | |||
| 106 | The query is performed via a call to pci_set_dma_mask(): | ||
| 107 | |||
| 108 | int pci_set_dma_mask(struct pci_dev *pdev, u64 device_mask); | ||
| 109 | |||
| 110 | The query for consistent allocations is performed via a call to | ||
| 111 | pci_set_consistent_dma_mask(): | ||
| 112 | |||
| 113 | int pci_set_consistent_dma_mask(struct pci_dev *pdev, u64 device_mask); | ||
| 114 | |||
| 115 | Here, pdev is a pointer to the PCI device struct of your device, and | ||
| 116 | device_mask is a bit mask describing which bits of a PCI address your | ||
| 117 | device supports. It returns zero if your card can perform DMA | ||
| 118 | properly on the machine given the address mask you provided. | ||
| 119 | |||
| 120 | If it returns non-zero, your device cannot perform DMA properly on | ||
| 121 | this platform, and attempting to do so will result in undefined | ||
| 122 | behavior. You must either use a different mask, or not use DMA. | ||
| 123 | |||
| 124 | This means that in the failure case, you have three options: | ||
| 125 | |||
| 126 | 1) Use another DMA mask, if possible (see below). | ||
| 127 | 2) Use some non-DMA mode for data transfer, if possible. | ||
| 128 | 3) Ignore this device and do not initialize it. | ||
| 129 | |||
| 130 | It is recommended that your driver print a kernel KERN_WARNING message | ||
| 131 | when you end up performing either #2 or #3. In this manner, if a user | ||
| 132 | of your driver reports that performance is bad or that the device is not | ||
| 133 | even detected, you can ask them for the kernel messages to find out | ||
| 134 | exactly why. | ||
| 135 | |||
| 136 | The standard 32-bit addressing PCI device would do something like | ||
| 137 | this: | ||
| 138 | |||
| 139 | if (pci_set_dma_mask(pdev, DMA_BIT_MASK(32))) { | ||
| 140 | printk(KERN_WARNING | ||
| 141 | "mydev: No suitable DMA available.\n"); | ||
| 142 | goto ignore_this_device; | ||
| 143 | } | ||
| 144 | |||
| 145 | Another common scenario is a 64-bit capable device. The approach | ||
| 146 | here is to try for 64-bit DAC addressing, but back down to a | ||
| 147 | 32-bit mask should that fail. The PCI platform code may fail the | ||
| 148 | 64-bit mask not because the platform is not capable of 64-bit | ||
| 149 | addressing. Rather, it may fail in this case simply because | ||
| 150 | 32-bit SAC addressing is done more efficiently than DAC addressing. | ||
| 151 | Sparc64 is one platform which behaves in this way. | ||
| 152 | |||
| 153 | Here is how you would handle a 64-bit capable device which can drive | ||
| 154 | all 64-bits when accessing streaming DMA: | ||
| 155 | |||
| 156 | int using_dac; | ||
| 157 | |||
| 158 | if (!pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) { | ||
| 159 | using_dac = 1; | ||
| 160 | } else if (!pci_set_dma_mask(pdev, DMA_BIT_MASK(32))) { | ||
| 161 | using_dac = 0; | ||
| 162 | } else { | ||
| 163 | printk(KERN_WARNING | ||
| 164 | "mydev: No suitable DMA available.\n"); | ||
| 165 | goto ignore_this_device; | ||
| 166 | } | ||
| 167 | |||
| 168 | If a card is capable of using 64-bit consistent allocations as well, | ||
| 169 | the case would look like this: | ||
| 170 | |||
| 171 | int using_dac, consistent_using_dac; | ||
| 172 | |||
| 173 | if (!pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) { | ||
| 174 | using_dac = 1; | ||
| 175 | consistent_using_dac = 1; | ||
| 176 | pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64)); | ||
| 177 | } else if (!pci_set_dma_mask(pdev, DMA_BIT_MASK(32))) { | ||
| 178 | using_dac = 0; | ||
| 179 | consistent_using_dac = 0; | ||
| 180 | pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32)); | ||
| 181 | } else { | ||
| 182 | printk(KERN_WARNING | ||
| 183 | "mydev: No suitable DMA available.\n"); | ||
| 184 | goto ignore_this_device; | ||
| 185 | } | ||
| 186 | |||
| 187 | pci_set_consistent_dma_mask() will always be able to set the same or a | ||
| 188 | smaller mask as pci_set_dma_mask(). However for the rare case that a | ||
| 189 | device driver only uses consistent allocations, one would have to | ||
| 190 | check the return value from pci_set_consistent_dma_mask(). | ||
| 191 | |||
| 192 | Finally, if your device can only drive the low 24-bits of | ||
| 193 | address during PCI bus mastering you might do something like: | ||
| 194 | |||
| 195 | if (pci_set_dma_mask(pdev, DMA_BIT_MASK(24))) { | ||
| 196 | printk(KERN_WARNING | ||
| 197 | "mydev: 24-bit DMA addressing not available.\n"); | ||
| 198 | goto ignore_this_device; | ||
| 199 | } | ||
| 200 | |||
| 201 | When pci_set_dma_mask() is successful, and returns zero, the PCI layer | ||
| 202 | saves away this mask you have provided. The PCI layer will use this | ||
| 203 | information later when you make DMA mappings. | ||
| 204 | |||
| 205 | There is a case which we are aware of at this time, which is worth | ||
| 206 | mentioning in this documentation. If your device supports multiple | ||
| 207 | functions (for example a sound card provides playback and record | ||
| 208 | functions) and the various different functions have _different_ | ||
| 209 | DMA addressing limitations, you may wish to probe each mask and | ||
| 210 | only provide the functionality which the machine can handle. It | ||
| 211 | is important that the last call to pci_set_dma_mask() be for the | ||
| 212 | most specific mask. | ||
| 213 | |||
| 214 | Here is pseudo-code showing how this might be done: | ||
| 215 | |||
| 216 | #define PLAYBACK_ADDRESS_BITS DMA_BIT_MASK(32) | ||
| 217 | #define RECORD_ADDRESS_BITS DMA_BIT_MASK(24) | ||
| 218 | |||
| 219 | struct my_sound_card *card; | ||
| 220 | struct pci_dev *pdev; | ||
| 221 | |||
| 222 | ... | ||
| 223 | if (!pci_set_dma_mask(pdev, PLAYBACK_ADDRESS_BITS)) { | ||
| 224 | card->playback_enabled = 1; | ||
| 225 | } else { | ||
| 226 | card->playback_enabled = 0; | ||
| 227 | printk(KERN_WARNING "%s: Playback disabled due to DMA limitations.\n", | ||
| 228 | card->name); | ||
| 229 | } | ||
| 230 | if (!pci_set_dma_mask(pdev, RECORD_ADDRESS_BITS)) { | ||
| 231 | card->record_enabled = 1; | ||
| 232 | } else { | ||
| 233 | card->record_enabled = 0; | ||
| 234 | printk(KERN_WARNING "%s: Record disabled due to DMA limitations.\n", | ||
| 235 | card->name); | ||
| 236 | } | ||
| 237 | |||
| 238 | A sound card was used as an example here because this genre of PCI | ||
| 239 | devices seems to be littered with ISA chips given a PCI front end, | ||
| 240 | and thus retaining the 16MB DMA addressing limitations of ISA. | ||
| 241 | |||
| 242 | Types of DMA mappings | ||
| 243 | |||
| 244 | There are two types of DMA mappings: | ||
| 245 | |||
| 246 | - Consistent DMA mappings which are usually mapped at driver | ||
| 247 | initialization, unmapped at the end and for which the hardware should | ||
| 248 | guarantee that the device and the CPU can access the data | ||
| 249 | in parallel and will see updates made by each other without any | ||
| 250 | explicit software flushing. | ||
| 251 | |||
| 252 | Think of "consistent" as "synchronous" or "coherent". | ||
| 253 | |||
| 254 | The current default is to return consistent memory in the low 32 | ||
| 255 | bits of the PCI bus space. However, for future compatibility you | ||
| 256 | should set the consistent mask even if this default is fine for your | ||
| 257 | driver. | ||
| 258 | |||
| 259 | Good examples of what to use consistent mappings for are: | ||
| 260 | |||
| 261 | - Network card DMA ring descriptors. | ||
| 262 | - SCSI adapter mailbox command data structures. | ||
| 263 | - Device firmware microcode executed out of | ||
| 264 | main memory. | ||
| 265 | |||
| 266 | The invariant these examples all require is that any CPU store | ||
| 267 | to memory is immediately visible to the device, and vice | ||
| 268 | versa. Consistent mappings guarantee this. | ||
| 269 | |||
| 270 | IMPORTANT: Consistent DMA memory does not preclude the usage of | ||
| 271 | proper memory barriers. The CPU may reorder stores to | ||
| 272 | consistent memory just as it may normal memory. Example: | ||
| 273 | if it is important for the device to see the first word | ||
| 274 | of a descriptor updated before the second, you must do | ||
| 275 | something like: | ||
| 276 | |||
| 277 | desc->word0 = address; | ||
| 278 | wmb(); | ||
| 279 | desc->word1 = DESC_VALID; | ||
| 280 | |||
| 281 | in order to get correct behavior on all platforms. | ||
| 282 | |||
| 283 | Also, on some platforms your driver may need to flush CPU write | ||
| 284 | buffers in much the same way as it needs to flush write buffers | ||
| 285 | found in PCI bridges (such as by reading a register's value | ||
| 286 | after writing it). | ||
| 287 | |||
| 288 | - Streaming DMA mappings which are usually mapped for one DMA transfer, | ||
| 289 | unmapped right after it (unless you use pci_dma_sync_* below) and for which | ||
| 290 | hardware can optimize for sequential accesses. | ||
| 291 | |||
| 292 | This of "streaming" as "asynchronous" or "outside the coherency | ||
| 293 | domain". | ||
| 294 | |||
| 295 | Good examples of what to use streaming mappings for are: | ||
| 296 | |||
| 297 | - Networking buffers transmitted/received by a device. | ||
| 298 | - Filesystem buffers written/read by a SCSI device. | ||
| 299 | |||
| 300 | The interfaces for using this type of mapping were designed in | ||
| 301 | such a way that an implementation can make whatever performance | ||
| 302 | optimizations the hardware allows. To this end, when using | ||
| 303 | such mappings you must be explicit about what you want to happen. | ||
| 304 | |||
| 305 | Neither type of DMA mapping has alignment restrictions that come | ||
| 306 | from PCI, although some devices may have such restrictions. | ||
| 307 | Also, systems with caches that aren't DMA-coherent will work better | ||
| 308 | when the underlying buffers don't share cache lines with other data. | ||
| 309 | |||
| 310 | |||
| 311 | Using Consistent DMA mappings. | ||
| 312 | |||
| 313 | To allocate and map large (PAGE_SIZE or so) consistent DMA regions, | ||
| 314 | you should do: | ||
| 315 | |||
| 316 | dma_addr_t dma_handle; | ||
| 317 | |||
| 318 | cpu_addr = pci_alloc_consistent(pdev, size, &dma_handle); | ||
| 319 | |||
| 320 | where pdev is a struct pci_dev *. This may be called in interrupt context. | ||
| 321 | You should use dma_alloc_coherent (see DMA-API.txt) for buses | ||
| 322 | where devices don't have struct pci_dev (like ISA, EISA). | ||
| 323 | |||
| 324 | This argument is needed because the DMA translations may be bus | ||
| 325 | specific (and often is private to the bus which the device is attached | ||
| 326 | to). | ||
| 327 | |||
| 328 | Size is the length of the region you want to allocate, in bytes. | ||
| 329 | |||
| 330 | This routine will allocate RAM for that region, so it acts similarly to | ||
| 331 | __get_free_pages (but takes size instead of a page order). If your | ||
| 332 | driver needs regions sized smaller than a page, you may prefer using | ||
| 333 | the pci_pool interface, described below. | ||
| 334 | |||
| 335 | The consistent DMA mapping interfaces, for non-NULL pdev, will by | ||
| 336 | default return a DMA address which is SAC (Single Address Cycle) | ||
| 337 | addressable. Even if the device indicates (via PCI dma mask) that it | ||
| 338 | may address the upper 32-bits and thus perform DAC cycles, consistent | ||
| 339 | allocation will only return > 32-bit PCI addresses for DMA if the | ||
| 340 | consistent dma mask has been explicitly changed via | ||
| 341 | pci_set_consistent_dma_mask(). This is true of the pci_pool interface | ||
| 342 | as well. | ||
| 343 | |||
| 344 | pci_alloc_consistent returns two values: the virtual address which you | ||
| 345 | can use to access it from the CPU and dma_handle which you pass to the | ||
| 346 | card. | ||
| 347 | |||
| 348 | The cpu return address and the DMA bus master address are both | ||
| 349 | guaranteed to be aligned to the smallest PAGE_SIZE order which | ||
| 350 | is greater than or equal to the requested size. This invariant | ||
| 351 | exists (for example) to guarantee that if you allocate a chunk | ||
| 352 | which is smaller than or equal to 64 kilobytes, the extent of the | ||
| 353 | buffer you receive will not cross a 64K boundary. | ||
| 354 | |||
| 355 | To unmap and free such a DMA region, you call: | ||
| 356 | |||
| 357 | pci_free_consistent(pdev, size, cpu_addr, dma_handle); | ||
| 358 | |||
| 359 | where pdev, size are the same as in the above call and cpu_addr and | ||
| 360 | dma_handle are the values pci_alloc_consistent returned to you. | ||
| 361 | This function may not be called in interrupt context. | ||
| 362 | |||
| 363 | If your driver needs lots of smaller memory regions, you can write | ||
| 364 | custom code to subdivide pages returned by pci_alloc_consistent, | ||
| 365 | or you can use the pci_pool API to do that. A pci_pool is like | ||
| 366 | a kmem_cache, but it uses pci_alloc_consistent not __get_free_pages. | ||
| 367 | Also, it understands common hardware constraints for alignment, | ||
| 368 | like queue heads needing to be aligned on N byte boundaries. | ||
| 369 | |||
| 370 | Create a pci_pool like this: | ||
| 371 | |||
| 372 | struct pci_pool *pool; | ||
| 373 | |||
| 374 | pool = pci_pool_create(name, pdev, size, align, alloc); | ||
| 375 | |||
| 376 | The "name" is for diagnostics (like a kmem_cache name); pdev and size | ||
| 377 | are as above. The device's hardware alignment requirement for this | ||
| 378 | type of data is "align" (which is expressed in bytes, and must be a | ||
| 379 | power of two). If your device has no boundary crossing restrictions, | ||
| 380 | pass 0 for alloc; passing 4096 says memory allocated from this pool | ||
| 381 | must not cross 4KByte boundaries (but at that time it may be better to | ||
| 382 | go for pci_alloc_consistent directly instead). | ||
| 383 | |||
| 384 | Allocate memory from a pci pool like this: | ||
| 385 | |||
| 386 | cpu_addr = pci_pool_alloc(pool, flags, &dma_handle); | ||
| 387 | |||
| 388 | flags are SLAB_KERNEL if blocking is permitted (not in_interrupt nor | ||
| 389 | holding SMP locks), SLAB_ATOMIC otherwise. Like pci_alloc_consistent, | ||
| 390 | this returns two values, cpu_addr and dma_handle. | ||
| 391 | |||
| 392 | Free memory that was allocated from a pci_pool like this: | ||
| 393 | |||
| 394 | pci_pool_free(pool, cpu_addr, dma_handle); | ||
| 395 | |||
| 396 | where pool is what you passed to pci_pool_alloc, and cpu_addr and | ||
| 397 | dma_handle are the values pci_pool_alloc returned. This function | ||
| 398 | may be called in interrupt context. | ||
| 399 | |||
| 400 | Destroy a pci_pool by calling: | ||
| 401 | |||
| 402 | pci_pool_destroy(pool); | ||
| 403 | |||
| 404 | Make sure you've called pci_pool_free for all memory allocated | ||
| 405 | from a pool before you destroy the pool. This function may not | ||
| 406 | be called in interrupt context. | ||
| 407 | |||
| 408 | DMA Direction | ||
| 409 | |||
| 410 | The interfaces described in subsequent portions of this document | ||
| 411 | take a DMA direction argument, which is an integer and takes on | ||
| 412 | one of the following values: | ||
| 413 | |||
| 414 | PCI_DMA_BIDIRECTIONAL | ||
| 415 | PCI_DMA_TODEVICE | ||
| 416 | PCI_DMA_FROMDEVICE | ||
| 417 | PCI_DMA_NONE | ||
| 418 | |||
| 419 | One should provide the exact DMA direction if you know it. | ||
| 420 | |||
| 421 | PCI_DMA_TODEVICE means "from main memory to the PCI device" | ||
| 422 | PCI_DMA_FROMDEVICE means "from the PCI device to main memory" | ||
| 423 | It is the direction in which the data moves during the DMA | ||
| 424 | transfer. | ||
| 425 | |||
| 426 | You are _strongly_ encouraged to specify this as precisely | ||
| 427 | as you possibly can. | ||
| 428 | |||
| 429 | If you absolutely cannot know the direction of the DMA transfer, | ||
| 430 | specify PCI_DMA_BIDIRECTIONAL. It means that the DMA can go in | ||
| 431 | either direction. The platform guarantees that you may legally | ||
| 432 | specify this, and that it will work, but this may be at the | ||
| 433 | cost of performance for example. | ||
| 434 | |||
| 435 | The value PCI_DMA_NONE is to be used for debugging. One can | ||
| 436 | hold this in a data structure before you come to know the | ||
| 437 | precise direction, and this will help catch cases where your | ||
| 438 | direction tracking logic has failed to set things up properly. | ||
| 439 | |||
| 440 | Another advantage of specifying this value precisely (outside of | ||
| 441 | potential platform-specific optimizations of such) is for debugging. | ||
| 442 | Some platforms actually have a write permission boolean which DMA | ||
| 443 | mappings can be marked with, much like page protections in the user | ||
| 444 | program address space. Such platforms can and do report errors in the | ||
| 445 | kernel logs when the PCI controller hardware detects violation of the | ||
| 446 | permission setting. | ||
| 447 | |||
| 448 | Only streaming mappings specify a direction, consistent mappings | ||
| 449 | implicitly have a direction attribute setting of | ||
| 450 | PCI_DMA_BIDIRECTIONAL. | ||
| 451 | |||
| 452 | The SCSI subsystem tells you the direction to use in the | ||
| 453 | 'sc_data_direction' member of the SCSI command your driver is | ||
| 454 | working on. | ||
| 455 | |||
| 456 | For Networking drivers, it's a rather simple affair. For transmit | ||
| 457 | packets, map/unmap them with the PCI_DMA_TODEVICE direction | ||
| 458 | specifier. For receive packets, just the opposite, map/unmap them | ||
| 459 | with the PCI_DMA_FROMDEVICE direction specifier. | ||
| 460 | |||
| 461 | Using Streaming DMA mappings | ||
| 462 | |||
| 463 | The streaming DMA mapping routines can be called from interrupt | ||
| 464 | context. There are two versions of each map/unmap, one which will | ||
| 465 | map/unmap a single memory region, and one which will map/unmap a | ||
| 466 | scatterlist. | ||
| 467 | |||
| 468 | To map a single region, you do: | ||
| 469 | |||
| 470 | struct pci_dev *pdev = mydev->pdev; | ||
| 471 | dma_addr_t dma_handle; | ||
| 472 | void *addr = buffer->ptr; | ||
| 473 | size_t size = buffer->len; | ||
| 474 | |||
| 475 | dma_handle = pci_map_single(pdev, addr, size, direction); | ||
| 476 | |||
| 477 | and to unmap it: | ||
| 478 | |||
| 479 | pci_unmap_single(pdev, dma_handle, size, direction); | ||
| 480 | |||
| 481 | You should call pci_unmap_single when the DMA activity is finished, e.g. | ||
| 482 | from the interrupt which told you that the DMA transfer is done. | ||
| 483 | |||
| 484 | Using cpu pointers like this for single mappings has a disadvantage, | ||
| 485 | you cannot reference HIGHMEM memory in this way. Thus, there is a | ||
| 486 | map/unmap interface pair akin to pci_{map,unmap}_single. These | ||
| 487 | interfaces deal with page/offset pairs instead of cpu pointers. | ||
| 488 | Specifically: | ||
| 489 | |||
| 490 | struct pci_dev *pdev = mydev->pdev; | ||
| 491 | dma_addr_t dma_handle; | ||
| 492 | struct page *page = buffer->page; | ||
| 493 | unsigned long offset = buffer->offset; | ||
| 494 | size_t size = buffer->len; | ||
| 495 | |||
| 496 | dma_handle = pci_map_page(pdev, page, offset, size, direction); | ||
| 497 | |||
| 498 | ... | ||
| 499 | |||
| 500 | pci_unmap_page(pdev, dma_handle, size, direction); | ||
| 501 | |||
| 502 | Here, "offset" means byte offset within the given page. | ||
| 503 | |||
| 504 | With scatterlists, you map a region gathered from several regions by: | ||
| 505 | |||
| 506 | int i, count = pci_map_sg(pdev, sglist, nents, direction); | ||
| 507 | struct scatterlist *sg; | ||
| 508 | |||
| 509 | for_each_sg(sglist, sg, count, i) { | ||
| 510 | hw_address[i] = sg_dma_address(sg); | ||
| 511 | hw_len[i] = sg_dma_len(sg); | ||
| 512 | } | ||
| 513 | |||
| 514 | where nents is the number of entries in the sglist. | ||
| 515 | |||
| 516 | The implementation is free to merge several consecutive sglist entries | ||
| 517 | into one (e.g. if DMA mapping is done with PAGE_SIZE granularity, any | ||
| 518 | consecutive sglist entries can be merged into one provided the first one | ||
| 519 | ends and the second one starts on a page boundary - in fact this is a huge | ||
| 520 | advantage for cards which either cannot do scatter-gather or have very | ||
| 521 | limited number of scatter-gather entries) and returns the actual number | ||
| 522 | of sg entries it mapped them to. On failure 0 is returned. | ||
| 523 | |||
| 524 | Then you should loop count times (note: this can be less than nents times) | ||
| 525 | and use sg_dma_address() and sg_dma_len() macros where you previously | ||
| 526 | accessed sg->address and sg->length as shown above. | ||
| 527 | |||
| 528 | To unmap a scatterlist, just call: | ||
| 529 | |||
| 530 | pci_unmap_sg(pdev, sglist, nents, direction); | ||
| 531 | |||
| 532 | Again, make sure DMA activity has already finished. | ||
| 533 | |||
| 534 | PLEASE NOTE: The 'nents' argument to the pci_unmap_sg call must be | ||
| 535 | the _same_ one you passed into the pci_map_sg call, | ||
| 536 | it should _NOT_ be the 'count' value _returned_ from the | ||
| 537 | pci_map_sg call. | ||
| 538 | |||
| 539 | Every pci_map_{single,sg} call should have its pci_unmap_{single,sg} | ||
| 540 | counterpart, because the bus address space is a shared resource (although | ||
| 541 | in some ports the mapping is per each BUS so less devices contend for the | ||
| 542 | same bus address space) and you could render the machine unusable by eating | ||
| 543 | all bus addresses. | ||
| 544 | |||
| 545 | If you need to use the same streaming DMA region multiple times and touch | ||
| 546 | the data in between the DMA transfers, the buffer needs to be synced | ||
| 547 | properly in order for the cpu and device to see the most uptodate and | ||
| 548 | correct copy of the DMA buffer. | ||
| 549 | |||
| 550 | So, firstly, just map it with pci_map_{single,sg}, and after each DMA | ||
| 551 | transfer call either: | ||
| 552 | |||
| 553 | pci_dma_sync_single_for_cpu(pdev, dma_handle, size, direction); | ||
| 554 | |||
| 555 | or: | ||
| 556 | |||
| 557 | pci_dma_sync_sg_for_cpu(pdev, sglist, nents, direction); | ||
| 558 | |||
| 559 | as appropriate. | ||
| 560 | |||
| 561 | Then, if you wish to let the device get at the DMA area again, | ||
| 562 | finish accessing the data with the cpu, and then before actually | ||
| 563 | giving the buffer to the hardware call either: | ||
| 564 | |||
| 565 | pci_dma_sync_single_for_device(pdev, dma_handle, size, direction); | ||
| 566 | |||
| 567 | or: | ||
| 568 | |||
| 569 | pci_dma_sync_sg_for_device(dev, sglist, nents, direction); | ||
| 570 | |||
| 571 | as appropriate. | ||
| 572 | |||
| 573 | After the last DMA transfer call one of the DMA unmap routines | ||
| 574 | pci_unmap_{single,sg}. If you don't touch the data from the first pci_map_* | ||
| 575 | call till pci_unmap_*, then you don't have to call the pci_dma_sync_* | ||
| 576 | routines at all. | ||
| 577 | |||
| 578 | Here is pseudo code which shows a situation in which you would need | ||
| 579 | to use the pci_dma_sync_*() interfaces. | ||
| 580 | |||
| 581 | my_card_setup_receive_buffer(struct my_card *cp, char *buffer, int len) | ||
| 582 | { | ||
| 583 | dma_addr_t mapping; | ||
| 584 | |||
| 585 | mapping = pci_map_single(cp->pdev, buffer, len, PCI_DMA_FROMDEVICE); | ||
| 586 | |||
| 587 | cp->rx_buf = buffer; | ||
| 588 | cp->rx_len = len; | ||
| 589 | cp->rx_dma = mapping; | ||
| 590 | |||
| 591 | give_rx_buf_to_card(cp); | ||
| 592 | } | ||
| 593 | |||
| 594 | ... | ||
| 595 | |||
| 596 | my_card_interrupt_handler(int irq, void *devid, struct pt_regs *regs) | ||
| 597 | { | ||
| 598 | struct my_card *cp = devid; | ||
| 599 | |||
| 600 | ... | ||
| 601 | if (read_card_status(cp) == RX_BUF_TRANSFERRED) { | ||
| 602 | struct my_card_header *hp; | ||
| 603 | |||
| 604 | /* Examine the header to see if we wish | ||
| 605 | * to accept the data. But synchronize | ||
| 606 | * the DMA transfer with the CPU first | ||
| 607 | * so that we see updated contents. | ||
| 608 | */ | ||
| 609 | pci_dma_sync_single_for_cpu(cp->pdev, cp->rx_dma, | ||
| 610 | cp->rx_len, | ||
| 611 | PCI_DMA_FROMDEVICE); | ||
| 612 | |||
| 613 | /* Now it is safe to examine the buffer. */ | ||
| 614 | hp = (struct my_card_header *) cp->rx_buf; | ||
| 615 | if (header_is_ok(hp)) { | ||
| 616 | pci_unmap_single(cp->pdev, cp->rx_dma, cp->rx_len, | ||
| 617 | PCI_DMA_FROMDEVICE); | ||
| 618 | pass_to_upper_layers(cp->rx_buf); | ||
| 619 | make_and_setup_new_rx_buf(cp); | ||
| 620 | } else { | ||
| 621 | /* Just sync the buffer and give it back | ||
| 622 | * to the card. | ||
| 623 | */ | ||
| 624 | pci_dma_sync_single_for_device(cp->pdev, | ||
| 625 | cp->rx_dma, | ||
| 626 | cp->rx_len, | ||
| 627 | PCI_DMA_FROMDEVICE); | ||
| 628 | give_rx_buf_to_card(cp); | ||
| 629 | } | ||
| 630 | } | ||
| 631 | } | ||
| 632 | |||
| 633 | Drivers converted fully to this interface should not use virt_to_bus any | ||
| 634 | longer, nor should they use bus_to_virt. Some drivers have to be changed a | ||
| 635 | little bit, because there is no longer an equivalent to bus_to_virt in the | ||
| 636 | dynamic DMA mapping scheme - you have to always store the DMA addresses | ||
| 637 | returned by the pci_alloc_consistent, pci_pool_alloc, and pci_map_single | ||
| 638 | calls (pci_map_sg stores them in the scatterlist itself if the platform | ||
| 639 | supports dynamic DMA mapping in hardware) in your driver structures and/or | ||
| 640 | in the card registers. | ||
| 641 | |||
| 642 | All PCI drivers should be using these interfaces with no exceptions. | ||
| 643 | It is planned to completely remove virt_to_bus() and bus_to_virt() as | ||
| 644 | they are entirely deprecated. Some ports already do not provide these | ||
| 645 | as it is impossible to correctly support them. | ||
| 646 | |||
| 647 | Optimizing Unmap State Space Consumption | ||
| 648 | |||
| 649 | On many platforms, pci_unmap_{single,page}() is simply a nop. | ||
| 650 | Therefore, keeping track of the mapping address and length is a waste | ||
| 651 | of space. Instead of filling your drivers up with ifdefs and the like | ||
| 652 | to "work around" this (which would defeat the whole purpose of a | ||
| 653 | portable API) the following facilities are provided. | ||
| 654 | |||
| 655 | Actually, instead of describing the macros one by one, we'll | ||
| 656 | transform some example code. | ||
| 657 | |||
| 658 | 1) Use DECLARE_PCI_UNMAP_{ADDR,LEN} in state saving structures. | ||
| 659 | Example, before: | ||
| 660 | |||
| 661 | struct ring_state { | ||
| 662 | struct sk_buff *skb; | ||
| 663 | dma_addr_t mapping; | ||
| 664 | __u32 len; | ||
| 665 | }; | ||
| 666 | |||
| 667 | after: | ||
| 668 | |||
| 669 | struct ring_state { | ||
| 670 | struct sk_buff *skb; | ||
| 671 | DECLARE_PCI_UNMAP_ADDR(mapping) | ||
| 672 | DECLARE_PCI_UNMAP_LEN(len) | ||
| 673 | }; | ||
| 674 | |||
| 675 | NOTE: DO NOT put a semicolon at the end of the DECLARE_*() | ||
| 676 | macro. | ||
| 677 | |||
| 678 | 2) Use pci_unmap_{addr,len}_set to set these values. | ||
| 679 | Example, before: | ||
| 680 | |||
| 681 | ringp->mapping = FOO; | ||
| 682 | ringp->len = BAR; | ||
| 683 | |||
| 684 | after: | ||
| 685 | |||
| 686 | pci_unmap_addr_set(ringp, mapping, FOO); | ||
| 687 | pci_unmap_len_set(ringp, len, BAR); | ||
| 688 | |||
| 689 | 3) Use pci_unmap_{addr,len} to access these values. | ||
| 690 | Example, before: | ||
| 691 | |||
| 692 | pci_unmap_single(pdev, ringp->mapping, ringp->len, | ||
| 693 | PCI_DMA_FROMDEVICE); | ||
| 694 | |||
| 695 | after: | ||
| 696 | |||
| 697 | pci_unmap_single(pdev, | ||
| 698 | pci_unmap_addr(ringp, mapping), | ||
| 699 | pci_unmap_len(ringp, len), | ||
| 700 | PCI_DMA_FROMDEVICE); | ||
| 701 | |||
| 702 | It really should be self-explanatory. We treat the ADDR and LEN | ||
| 703 | separately, because it is possible for an implementation to only | ||
| 704 | need the address in order to perform the unmap operation. | ||
| 705 | |||
| 706 | Platform Issues | ||
| 707 | |||
| 708 | If you are just writing drivers for Linux and do not maintain | ||
| 709 | an architecture port for the kernel, you can safely skip down | ||
| 710 | to "Closing". | ||
| 711 | |||
| 712 | 1) Struct scatterlist requirements. | ||
| 713 | |||
| 714 | Struct scatterlist must contain, at a minimum, the following | ||
| 715 | members: | ||
| 716 | |||
| 717 | struct page *page; | ||
| 718 | unsigned int offset; | ||
| 719 | unsigned int length; | ||
| 720 | |||
| 721 | The base address is specified by a "page+offset" pair. | ||
| 722 | |||
| 723 | Previous versions of struct scatterlist contained a "void *address" | ||
| 724 | field that was sometimes used instead of page+offset. As of Linux | ||
| 725 | 2.5., page+offset is always used, and the "address" field has been | ||
| 726 | deleted. | ||
| 727 | |||
| 728 | 2) More to come... | ||
| 729 | |||
| 730 | Handling Errors | ||
| 731 | |||
| 732 | DMA address space is limited on some architectures and an allocation | ||
| 733 | failure can be determined by: | ||
| 734 | |||
| 735 | - checking if pci_alloc_consistent returns NULL or pci_map_sg returns 0 | ||
| 736 | |||
| 737 | - checking the returned dma_addr_t of pci_map_single and pci_map_page | ||
| 738 | by using pci_dma_mapping_error(): | ||
| 739 | |||
| 740 | dma_addr_t dma_handle; | ||
| 741 | |||
| 742 | dma_handle = pci_map_single(pdev, addr, size, direction); | ||
| 743 | if (pci_dma_mapping_error(pdev, dma_handle)) { | ||
| 744 | /* | ||
| 745 | * reduce current DMA mapping usage, | ||
| 746 | * delay and try again later or | ||
| 747 | * reset driver. | ||
| 748 | */ | ||
| 749 | } | ||
| 750 | |||
| 751 | Closing | ||
| 752 | |||
| 753 | This document, and the API itself, would not be in it's current | ||
| 754 | form without the feedback and suggestions from numerous individuals. | ||
| 755 | We would like to specifically mention, in no particular order, the | ||
| 756 | following people: | ||
| 757 | |||
| 758 | Russell King <rmk@arm.linux.org.uk> | ||
| 759 | Leo Dagum <dagum@barrel.engr.sgi.com> | ||
| 760 | Ralf Baechle <ralf@oss.sgi.com> | ||
| 761 | Grant Grundler <grundler@cup.hp.com> | ||
| 762 | Jay Estabrook <Jay.Estabrook@compaq.com> | ||
| 763 | Thomas Sailer <sailer@ife.ee.ethz.ch> | ||
| 764 | Andrea Arcangeli <andrea@suse.de> | ||
| 765 | Jens Axboe <jens.axboe@oracle.com> | ||
| 766 | David Mosberger-Tang <davidm@hpl.hp.com> | ||
