diff options
author | Linus Torvalds <torvalds@ppc970.osdl.org> | 2005-04-16 18:20:36 -0400 |
---|---|---|
committer | Linus Torvalds <torvalds@ppc970.osdl.org> | 2005-04-16 18:20:36 -0400 |
commit | 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2 (patch) | |
tree | 0bba044c4ce775e45a88a51686b5d9f90697ea9d /Documentation/DMA-mapping.txt |
Linux-2.6.12-rc2v2.6.12-rc2
Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.
Let it rip!
Diffstat (limited to 'Documentation/DMA-mapping.txt')
-rw-r--r-- | Documentation/DMA-mapping.txt | 881 |
1 files changed, 881 insertions, 0 deletions
diff --git a/Documentation/DMA-mapping.txt b/Documentation/DMA-mapping.txt new file mode 100644 index 000000000000..f4ac37f157ea --- /dev/null +++ b/Documentation/DMA-mapping.txt | |||
@@ -0,0 +1,881 @@ | |||
1 | Dynamic DMA mapping | ||
2 | =================== | ||
3 | |||
4 | David S. Miller <davem@redhat.com> | ||
5 | Richard Henderson <rth@cygnus.com> | ||
6 | Jakub Jelinek <jakub@redhat.com> | ||
7 | |||
8 | This document describes the DMA mapping system in terms of the pci_ | ||
9 | API. For a similar API that works for generic devices, see | ||
10 | DMA-API.txt. | ||
11 | |||
12 | Most of the 64bit platforms have special hardware that translates bus | ||
13 | addresses (DMA addresses) into physical addresses. This is similar to | ||
14 | how page tables and/or a TLB translates virtual addresses to physical | ||
15 | addresses on a CPU. This is needed so that e.g. PCI devices can | ||
16 | access with a Single Address Cycle (32bit DMA address) any page in the | ||
17 | 64bit physical address space. Previously in Linux those 64bit | ||
18 | platforms had to set artificial limits on the maximum RAM size in the | ||
19 | system, so that the virt_to_bus() static scheme works (the DMA address | ||
20 | translation tables were simply filled on bootup to map each bus | ||
21 | address to the physical page __pa(bus_to_virt())). | ||
22 | |||
23 | So that Linux can use the dynamic DMA mapping, it needs some help from the | ||
24 | drivers, namely it has to take into account that DMA addresses should be | ||
25 | mapped only for the time they are actually used and unmapped after the DMA | ||
26 | transfer. | ||
27 | |||
28 | The following API will work of course even on platforms where no such | ||
29 | hardware exists, see e.g. include/asm-i386/pci.h for how it is implemented on | ||
30 | top of the virt_to_bus interface. | ||
31 | |||
32 | First of all, you should make sure | ||
33 | |||
34 | #include <linux/pci.h> | ||
35 | |||
36 | is in your driver. This file will obtain for you the definition of the | ||
37 | dma_addr_t (which can hold any valid DMA address for the platform) | ||
38 | type which should be used everywhere you hold a DMA (bus) address | ||
39 | returned from the DMA mapping functions. | ||
40 | |||
41 | What memory is DMA'able? | ||
42 | |||
43 | The first piece of information you must know is what kernel memory can | ||
44 | be used with the DMA mapping facilities. There has been an unwritten | ||
45 | set of rules regarding this, and this text is an attempt to finally | ||
46 | write them down. | ||
47 | |||
48 | If you acquired your memory via the page allocator | ||
49 | (i.e. __get_free_page*()) or the generic memory allocators | ||
50 | (i.e. kmalloc() or kmem_cache_alloc()) then you may DMA to/from | ||
51 | that memory using the addresses returned from those routines. | ||
52 | |||
53 | This means specifically that you may _not_ use the memory/addresses | ||
54 | returned from vmalloc() for DMA. It is possible to DMA to the | ||
55 | _underlying_ memory mapped into a vmalloc() area, but this requires | ||
56 | walking page tables to get the physical addresses, and then | ||
57 | translating each of those pages back to a kernel address using | ||
58 | something like __va(). [ EDIT: Update this when we integrate | ||
59 | Gerd Knorr's generic code which does this. ] | ||
60 | |||
61 | This rule also means that you may not use kernel image addresses | ||
62 | (ie. items in the kernel's data/text/bss segment, or your driver's) | ||
63 | nor may you use kernel stack addresses for DMA. Both of these items | ||
64 | might be mapped somewhere entirely different than the rest of physical | ||
65 | memory. | ||
66 | |||
67 | Also, this means that you cannot take the return of a kmap() | ||
68 | call and DMA to/from that. This is similar to vmalloc(). | ||
69 | |||
70 | What about block I/O and networking buffers? The block I/O and | ||
71 | networking subsystems make sure that the buffers they use are valid | ||
72 | for you to DMA from/to. | ||
73 | |||
74 | DMA addressing limitations | ||
75 | |||
76 | Does your device have any DMA addressing limitations? For example, is | ||
77 | your device only capable of driving the low order 24-bits of address | ||
78 | on the PCI bus for SAC DMA transfers? If so, you need to inform the | ||
79 | PCI layer of this fact. | ||
80 | |||
81 | By default, the kernel assumes that your device can address the full | ||
82 | 32-bits in a SAC cycle. For a 64-bit DAC capable device, this needs | ||
83 | to be increased. And for a device with limitations, as discussed in | ||
84 | the previous paragraph, it needs to be decreased. | ||
85 | |||
86 | pci_alloc_consistent() by default will return 32-bit DMA addresses. | ||
87 | PCI-X specification requires PCI-X devices to support 64-bit | ||
88 | addressing (DAC) for all transactions. And at least one platform (SGI | ||
89 | SN2) requires 64-bit consistent allocations to operate correctly when | ||
90 | the IO bus is in PCI-X mode. Therefore, like with pci_set_dma_mask(), | ||
91 | it's good practice to call pci_set_consistent_dma_mask() to set the | ||
92 | appropriate mask even if your device only supports 32-bit DMA | ||
93 | (default) and especially if it's a PCI-X device. | ||
94 | |||
95 | For correct operation, you must interrogate the PCI layer in your | ||
96 | device probe routine to see if the PCI controller on the machine can | ||
97 | properly support the DMA addressing limitation your device has. It is | ||
98 | good style to do this even if your device holds the default setting, | ||
99 | because this shows that you did think about these issues wrt. your | ||
100 | device. | ||
101 | |||
102 | The query is performed via a call to pci_set_dma_mask(): | ||
103 | |||
104 | int pci_set_dma_mask(struct pci_dev *pdev, u64 device_mask); | ||
105 | |||
106 | The query for consistent allocations is performed via a a call to | ||
107 | pci_set_consistent_dma_mask(): | ||
108 | |||
109 | int pci_set_consistent_dma_mask(struct pci_dev *pdev, u64 device_mask); | ||
110 | |||
111 | Here, pdev is a pointer to the PCI device struct of your device, and | ||
112 | device_mask is a bit mask describing which bits of a PCI address your | ||
113 | device supports. It returns zero if your card can perform DMA | ||
114 | properly on the machine given the address mask you provided. | ||
115 | |||
116 | If it returns non-zero, your device can not perform DMA properly on | ||
117 | this platform, and attempting to do so will result in undefined | ||
118 | behavior. You must either use a different mask, or not use DMA. | ||
119 | |||
120 | This means that in the failure case, you have three options: | ||
121 | |||
122 | 1) Use another DMA mask, if possible (see below). | ||
123 | 2) Use some non-DMA mode for data transfer, if possible. | ||
124 | 3) Ignore this device and do not initialize it. | ||
125 | |||
126 | It is recommended that your driver print a kernel KERN_WARNING message | ||
127 | when you end up performing either #2 or #3. In this manner, if a user | ||
128 | of your driver reports that performance is bad or that the device is not | ||
129 | even detected, you can ask them for the kernel messages to find out | ||
130 | exactly why. | ||
131 | |||
132 | The standard 32-bit addressing PCI device would do something like | ||
133 | this: | ||
134 | |||
135 | if (pci_set_dma_mask(pdev, DMA_32BIT_MASK)) { | ||
136 | printk(KERN_WARNING | ||
137 | "mydev: No suitable DMA available.\n"); | ||
138 | goto ignore_this_device; | ||
139 | } | ||
140 | |||
141 | Another common scenario is a 64-bit capable device. The approach | ||
142 | here is to try for 64-bit DAC addressing, but back down to a | ||
143 | 32-bit mask should that fail. The PCI platform code may fail the | ||
144 | 64-bit mask not because the platform is not capable of 64-bit | ||
145 | addressing. Rather, it may fail in this case simply because | ||
146 | 32-bit SAC addressing is done more efficiently than DAC addressing. | ||
147 | Sparc64 is one platform which behaves in this way. | ||
148 | |||
149 | Here is how you would handle a 64-bit capable device which can drive | ||
150 | all 64-bits when accessing streaming DMA: | ||
151 | |||
152 | int using_dac; | ||
153 | |||
154 | if (!pci_set_dma_mask(pdev, DMA_64BIT_MASK)) { | ||
155 | using_dac = 1; | ||
156 | } else if (!pci_set_dma_mask(pdev, DMA_32BIT_MASK)) { | ||
157 | using_dac = 0; | ||
158 | } else { | ||
159 | printk(KERN_WARNING | ||
160 | "mydev: No suitable DMA available.\n"); | ||
161 | goto ignore_this_device; | ||
162 | } | ||
163 | |||
164 | If a card is capable of using 64-bit consistent allocations as well, | ||
165 | the case would look like this: | ||
166 | |||
167 | int using_dac, consistent_using_dac; | ||
168 | |||
169 | if (!pci_set_dma_mask(pdev, DMA_64BIT_MASK)) { | ||
170 | using_dac = 1; | ||
171 | consistent_using_dac = 1; | ||
172 | pci_set_consistent_dma_mask(pdev, DMA_64BIT_MASK); | ||
173 | } else if (!pci_set_dma_mask(pdev, DMA_32BIT_MASK)) { | ||
174 | using_dac = 0; | ||
175 | consistent_using_dac = 0; | ||
176 | pci_set_consistent_dma_mask(pdev, DMA_32BIT_MASK); | ||
177 | } else { | ||
178 | printk(KERN_WARNING | ||
179 | "mydev: No suitable DMA available.\n"); | ||
180 | goto ignore_this_device; | ||
181 | } | ||
182 | |||
183 | pci_set_consistent_dma_mask() will always be able to set the same or a | ||
184 | smaller mask as pci_set_dma_mask(). However for the rare case that a | ||
185 | device driver only uses consistent allocations, one would have to | ||
186 | check the return value from pci_set_consistent_dma_mask(). | ||
187 | |||
188 | If your 64-bit device is going to be an enormous consumer of DMA | ||
189 | mappings, this can be problematic since the DMA mappings are a | ||
190 | finite resource on many platforms. Please see the "DAC Addressing | ||
191 | for Address Space Hungry Devices" section near the end of this | ||
192 | document for how to handle this case. | ||
193 | |||
194 | Finally, if your device can only drive the low 24-bits of | ||
195 | address during PCI bus mastering you might do something like: | ||
196 | |||
197 | if (pci_set_dma_mask(pdev, 0x00ffffff)) { | ||
198 | printk(KERN_WARNING | ||
199 | "mydev: 24-bit DMA addressing not available.\n"); | ||
200 | goto ignore_this_device; | ||
201 | } | ||
202 | |||
203 | When pci_set_dma_mask() is successful, and returns zero, the PCI layer | ||
204 | saves away this mask you have provided. The PCI layer will use this | ||
205 | information later when you make DMA mappings. | ||
206 | |||
207 | There is a case which we are aware of at this time, which is worth | ||
208 | mentioning in this documentation. If your device supports multiple | ||
209 | functions (for example a sound card provides playback and record | ||
210 | functions) and the various different functions have _different_ | ||
211 | DMA addressing limitations, you may wish to probe each mask and | ||
212 | only provide the functionality which the machine can handle. It | ||
213 | is important that the last call to pci_set_dma_mask() be for the | ||
214 | most specific mask. | ||
215 | |||
216 | Here is pseudo-code showing how this might be done: | ||
217 | |||
218 | #define PLAYBACK_ADDRESS_BITS DMA_32BIT_MASK | ||
219 | #define RECORD_ADDRESS_BITS 0x00ffffff | ||
220 | |||
221 | struct my_sound_card *card; | ||
222 | struct pci_dev *pdev; | ||
223 | |||
224 | ... | ||
225 | if (!pci_set_dma_mask(pdev, PLAYBACK_ADDRESS_BITS)) { | ||
226 | card->playback_enabled = 1; | ||
227 | } else { | ||
228 | card->playback_enabled = 0; | ||
229 | printk(KERN_WARN "%s: Playback disabled due to DMA limitations.\n", | ||
230 | card->name); | ||
231 | } | ||
232 | if (!pci_set_dma_mask(pdev, RECORD_ADDRESS_BITS)) { | ||
233 | card->record_enabled = 1; | ||
234 | } else { | ||
235 | card->record_enabled = 0; | ||
236 | printk(KERN_WARN "%s: Record disabled due to DMA limitations.\n", | ||
237 | card->name); | ||
238 | } | ||
239 | |||
240 | A sound card was used as an example here because this genre of PCI | ||
241 | devices seems to be littered with ISA chips given a PCI front end, | ||
242 | and thus retaining the 16MB DMA addressing limitations of ISA. | ||
243 | |||
244 | Types of DMA mappings | ||
245 | |||
246 | There are two types of DMA mappings: | ||
247 | |||
248 | - Consistent DMA mappings which are usually mapped at driver | ||
249 | initialization, unmapped at the end and for which the hardware should | ||
250 | guarantee that the device and the CPU can access the data | ||
251 | in parallel and will see updates made by each other without any | ||
252 | explicit software flushing. | ||
253 | |||
254 | Think of "consistent" as "synchronous" or "coherent". | ||
255 | |||
256 | The current default is to return consistent memory in the low 32 | ||
257 | bits of the PCI bus space. However, for future compatibility you | ||
258 | should set the consistent mask even if this default is fine for your | ||
259 | driver. | ||
260 | |||
261 | Good examples of what to use consistent mappings for are: | ||
262 | |||
263 | - Network card DMA ring descriptors. | ||
264 | - SCSI adapter mailbox command data structures. | ||
265 | - Device firmware microcode executed out of | ||
266 | main memory. | ||
267 | |||
268 | The invariant these examples all require is that any CPU store | ||
269 | to memory is immediately visible to the device, and vice | ||
270 | versa. Consistent mappings guarantee this. | ||
271 | |||
272 | IMPORTANT: Consistent DMA memory does not preclude the usage of | ||
273 | proper memory barriers. The CPU may reorder stores to | ||
274 | consistent memory just as it may normal memory. Example: | ||
275 | if it is important for the device to see the first word | ||
276 | of a descriptor updated before the second, you must do | ||
277 | something like: | ||
278 | |||
279 | desc->word0 = address; | ||
280 | wmb(); | ||
281 | desc->word1 = DESC_VALID; | ||
282 | |||
283 | in order to get correct behavior on all platforms. | ||
284 | |||
285 | - Streaming DMA mappings which are usually mapped for one DMA transfer, | ||
286 | unmapped right after it (unless you use pci_dma_sync_* below) and for which | ||
287 | hardware can optimize for sequential accesses. | ||
288 | |||
289 | This of "streaming" as "asynchronous" or "outside the coherency | ||
290 | domain". | ||
291 | |||
292 | Good examples of what to use streaming mappings for are: | ||
293 | |||
294 | - Networking buffers transmitted/received by a device. | ||
295 | - Filesystem buffers written/read by a SCSI device. | ||
296 | |||
297 | The interfaces for using this type of mapping were designed in | ||
298 | such a way that an implementation can make whatever performance | ||
299 | optimizations the hardware allows. To this end, when using | ||
300 | such mappings you must be explicit about what you want to happen. | ||
301 | |||
302 | Neither type of DMA mapping has alignment restrictions that come | ||
303 | from PCI, although some devices may have such restrictions. | ||
304 | |||
305 | Using Consistent DMA mappings. | ||
306 | |||
307 | To allocate and map large (PAGE_SIZE or so) consistent DMA regions, | ||
308 | you should do: | ||
309 | |||
310 | dma_addr_t dma_handle; | ||
311 | |||
312 | cpu_addr = pci_alloc_consistent(dev, size, &dma_handle); | ||
313 | |||
314 | where dev is a struct pci_dev *. You should pass NULL for PCI like buses | ||
315 | where devices don't have struct pci_dev (like ISA, EISA). This may be | ||
316 | called in interrupt context. | ||
317 | |||
318 | This argument is needed because the DMA translations may be bus | ||
319 | specific (and often is private to the bus which the device is attached | ||
320 | to). | ||
321 | |||
322 | Size is the length of the region you want to allocate, in bytes. | ||
323 | |||
324 | This routine will allocate RAM for that region, so it acts similarly to | ||
325 | __get_free_pages (but takes size instead of a page order). If your | ||
326 | driver needs regions sized smaller than a page, you may prefer using | ||
327 | the pci_pool interface, described below. | ||
328 | |||
329 | The consistent DMA mapping interfaces, for non-NULL dev, will by | ||
330 | default return a DMA address which is SAC (Single Address Cycle) | ||
331 | addressable. Even if the device indicates (via PCI dma mask) that it | ||
332 | may address the upper 32-bits and thus perform DAC cycles, consistent | ||
333 | allocation will only return > 32-bit PCI addresses for DMA if the | ||
334 | consistent dma mask has been explicitly changed via | ||
335 | pci_set_consistent_dma_mask(). This is true of the pci_pool interface | ||
336 | as well. | ||
337 | |||
338 | pci_alloc_consistent returns two values: the virtual address which you | ||
339 | can use to access it from the CPU and dma_handle which you pass to the | ||
340 | card. | ||
341 | |||
342 | The cpu return address and the DMA bus master address are both | ||
343 | guaranteed to be aligned to the smallest PAGE_SIZE order which | ||
344 | is greater than or equal to the requested size. This invariant | ||
345 | exists (for example) to guarantee that if you allocate a chunk | ||
346 | which is smaller than or equal to 64 kilobytes, the extent of the | ||
347 | buffer you receive will not cross a 64K boundary. | ||
348 | |||
349 | To unmap and free such a DMA region, you call: | ||
350 | |||
351 | pci_free_consistent(dev, size, cpu_addr, dma_handle); | ||
352 | |||
353 | where dev, size are the same as in the above call and cpu_addr and | ||
354 | dma_handle are the values pci_alloc_consistent returned to you. | ||
355 | This function may not be called in interrupt context. | ||
356 | |||
357 | If your driver needs lots of smaller memory regions, you can write | ||
358 | custom code to subdivide pages returned by pci_alloc_consistent, | ||
359 | or you can use the pci_pool API to do that. A pci_pool is like | ||
360 | a kmem_cache, but it uses pci_alloc_consistent not __get_free_pages. | ||
361 | Also, it understands common hardware constraints for alignment, | ||
362 | like queue heads needing to be aligned on N byte boundaries. | ||
363 | |||
364 | Create a pci_pool like this: | ||
365 | |||
366 | struct pci_pool *pool; | ||
367 | |||
368 | pool = pci_pool_create(name, dev, size, align, alloc); | ||
369 | |||
370 | The "name" is for diagnostics (like a kmem_cache name); dev and size | ||
371 | are as above. The device's hardware alignment requirement for this | ||
372 | type of data is "align" (which is expressed in bytes, and must be a | ||
373 | power of two). If your device has no boundary crossing restrictions, | ||
374 | pass 0 for alloc; passing 4096 says memory allocated from this pool | ||
375 | must not cross 4KByte boundaries (but at that time it may be better to | ||
376 | go for pci_alloc_consistent directly instead). | ||
377 | |||
378 | Allocate memory from a pci pool like this: | ||
379 | |||
380 | cpu_addr = pci_pool_alloc(pool, flags, &dma_handle); | ||
381 | |||
382 | flags are SLAB_KERNEL if blocking is permitted (not in_interrupt nor | ||
383 | holding SMP locks), SLAB_ATOMIC otherwise. Like pci_alloc_consistent, | ||
384 | this returns two values, cpu_addr and dma_handle. | ||
385 | |||
386 | Free memory that was allocated from a pci_pool like this: | ||
387 | |||
388 | pci_pool_free(pool, cpu_addr, dma_handle); | ||
389 | |||
390 | where pool is what you passed to pci_pool_alloc, and cpu_addr and | ||
391 | dma_handle are the values pci_pool_alloc returned. This function | ||
392 | may be called in interrupt context. | ||
393 | |||
394 | Destroy a pci_pool by calling: | ||
395 | |||
396 | pci_pool_destroy(pool); | ||
397 | |||
398 | Make sure you've called pci_pool_free for all memory allocated | ||
399 | from a pool before you destroy the pool. This function may not | ||
400 | be called in interrupt context. | ||
401 | |||
402 | DMA Direction | ||
403 | |||
404 | The interfaces described in subsequent portions of this document | ||
405 | take a DMA direction argument, which is an integer and takes on | ||
406 | one of the following values: | ||
407 | |||
408 | PCI_DMA_BIDIRECTIONAL | ||
409 | PCI_DMA_TODEVICE | ||
410 | PCI_DMA_FROMDEVICE | ||
411 | PCI_DMA_NONE | ||
412 | |||
413 | One should provide the exact DMA direction if you know it. | ||
414 | |||
415 | PCI_DMA_TODEVICE means "from main memory to the PCI device" | ||
416 | PCI_DMA_FROMDEVICE means "from the PCI device to main memory" | ||
417 | It is the direction in which the data moves during the DMA | ||
418 | transfer. | ||
419 | |||
420 | You are _strongly_ encouraged to specify this as precisely | ||
421 | as you possibly can. | ||
422 | |||
423 | If you absolutely cannot know the direction of the DMA transfer, | ||
424 | specify PCI_DMA_BIDIRECTIONAL. It means that the DMA can go in | ||
425 | either direction. The platform guarantees that you may legally | ||
426 | specify this, and that it will work, but this may be at the | ||
427 | cost of performance for example. | ||
428 | |||
429 | The value PCI_DMA_NONE is to be used for debugging. One can | ||
430 | hold this in a data structure before you come to know the | ||
431 | precise direction, and this will help catch cases where your | ||
432 | direction tracking logic has failed to set things up properly. | ||
433 | |||
434 | Another advantage of specifying this value precisely (outside of | ||
435 | potential platform-specific optimizations of such) is for debugging. | ||
436 | Some platforms actually have a write permission boolean which DMA | ||
437 | mappings can be marked with, much like page protections in the user | ||
438 | program address space. Such platforms can and do report errors in the | ||
439 | kernel logs when the PCI controller hardware detects violation of the | ||
440 | permission setting. | ||
441 | |||
442 | Only streaming mappings specify a direction, consistent mappings | ||
443 | implicitly have a direction attribute setting of | ||
444 | PCI_DMA_BIDIRECTIONAL. | ||
445 | |||
446 | The SCSI subsystem provides mechanisms for you to easily obtain | ||
447 | the direction to use, in the SCSI command: | ||
448 | |||
449 | scsi_to_pci_dma_dir(SCSI_DIRECTION) | ||
450 | |||
451 | Where SCSI_DIRECTION is obtained from the 'sc_data_direction' | ||
452 | member of the SCSI command your driver is working on. The | ||
453 | mentioned interface above returns a value suitable for passing | ||
454 | into the streaming DMA mapping interfaces below. | ||
455 | |||
456 | For Networking drivers, it's a rather simple affair. For transmit | ||
457 | packets, map/unmap them with the PCI_DMA_TODEVICE direction | ||
458 | specifier. For receive packets, just the opposite, map/unmap them | ||
459 | with the PCI_DMA_FROMDEVICE direction specifier. | ||
460 | |||
461 | Using Streaming DMA mappings | ||
462 | |||
463 | The streaming DMA mapping routines can be called from interrupt | ||
464 | context. There are two versions of each map/unmap, one which will | ||
465 | map/unmap a single memory region, and one which will map/unmap a | ||
466 | scatterlist. | ||
467 | |||
468 | To map a single region, you do: | ||
469 | |||
470 | struct pci_dev *pdev = mydev->pdev; | ||
471 | dma_addr_t dma_handle; | ||
472 | void *addr = buffer->ptr; | ||
473 | size_t size = buffer->len; | ||
474 | |||
475 | dma_handle = pci_map_single(dev, addr, size, direction); | ||
476 | |||
477 | and to unmap it: | ||
478 | |||
479 | pci_unmap_single(dev, dma_handle, size, direction); | ||
480 | |||
481 | You should call pci_unmap_single when the DMA activity is finished, e.g. | ||
482 | from the interrupt which told you that the DMA transfer is done. | ||
483 | |||
484 | Using cpu pointers like this for single mappings has a disadvantage, | ||
485 | you cannot reference HIGHMEM memory in this way. Thus, there is a | ||
486 | map/unmap interface pair akin to pci_{map,unmap}_single. These | ||
487 | interfaces deal with page/offset pairs instead of cpu pointers. | ||
488 | Specifically: | ||
489 | |||
490 | struct pci_dev *pdev = mydev->pdev; | ||
491 | dma_addr_t dma_handle; | ||
492 | struct page *page = buffer->page; | ||
493 | unsigned long offset = buffer->offset; | ||
494 | size_t size = buffer->len; | ||
495 | |||
496 | dma_handle = pci_map_page(dev, page, offset, size, direction); | ||
497 | |||
498 | ... | ||
499 | |||
500 | pci_unmap_page(dev, dma_handle, size, direction); | ||
501 | |||
502 | Here, "offset" means byte offset within the given page. | ||
503 | |||
504 | With scatterlists, you map a region gathered from several regions by: | ||
505 | |||
506 | int i, count = pci_map_sg(dev, sglist, nents, direction); | ||
507 | struct scatterlist *sg; | ||
508 | |||
509 | for (i = 0, sg = sglist; i < count; i++, sg++) { | ||
510 | hw_address[i] = sg_dma_address(sg); | ||
511 | hw_len[i] = sg_dma_len(sg); | ||
512 | } | ||
513 | |||
514 | where nents is the number of entries in the sglist. | ||
515 | |||
516 | The implementation is free to merge several consecutive sglist entries | ||
517 | into one (e.g. if DMA mapping is done with PAGE_SIZE granularity, any | ||
518 | consecutive sglist entries can be merged into one provided the first one | ||
519 | ends and the second one starts on a page boundary - in fact this is a huge | ||
520 | advantage for cards which either cannot do scatter-gather or have very | ||
521 | limited number of scatter-gather entries) and returns the actual number | ||
522 | of sg entries it mapped them to. On failure 0 is returned. | ||
523 | |||
524 | Then you should loop count times (note: this can be less than nents times) | ||
525 | and use sg_dma_address() and sg_dma_len() macros where you previously | ||
526 | accessed sg->address and sg->length as shown above. | ||
527 | |||
528 | To unmap a scatterlist, just call: | ||
529 | |||
530 | pci_unmap_sg(dev, sglist, nents, direction); | ||
531 | |||
532 | Again, make sure DMA activity has already finished. | ||
533 | |||
534 | PLEASE NOTE: The 'nents' argument to the pci_unmap_sg call must be | ||
535 | the _same_ one you passed into the pci_map_sg call, | ||
536 | it should _NOT_ be the 'count' value _returned_ from the | ||
537 | pci_map_sg call. | ||
538 | |||
539 | Every pci_map_{single,sg} call should have its pci_unmap_{single,sg} | ||
540 | counterpart, because the bus address space is a shared resource (although | ||
541 | in some ports the mapping is per each BUS so less devices contend for the | ||
542 | same bus address space) and you could render the machine unusable by eating | ||
543 | all bus addresses. | ||
544 | |||
545 | If you need to use the same streaming DMA region multiple times and touch | ||
546 | the data in between the DMA transfers, the buffer needs to be synced | ||
547 | properly in order for the cpu and device to see the most uptodate and | ||
548 | correct copy of the DMA buffer. | ||
549 | |||
550 | So, firstly, just map it with pci_map_{single,sg}, and after each DMA | ||
551 | transfer call either: | ||
552 | |||
553 | pci_dma_sync_single_for_cpu(dev, dma_handle, size, direction); | ||
554 | |||
555 | or: | ||
556 | |||
557 | pci_dma_sync_sg_for_cpu(dev, sglist, nents, direction); | ||
558 | |||
559 | as appropriate. | ||
560 | |||
561 | Then, if you wish to let the device get at the DMA area again, | ||
562 | finish accessing the data with the cpu, and then before actually | ||
563 | giving the buffer to the hardware call either: | ||
564 | |||
565 | pci_dma_sync_single_for_device(dev, dma_handle, size, direction); | ||
566 | |||
567 | or: | ||
568 | |||
569 | pci_dma_sync_sg_for_device(dev, sglist, nents, direction); | ||
570 | |||
571 | as appropriate. | ||
572 | |||
573 | After the last DMA transfer call one of the DMA unmap routines | ||
574 | pci_unmap_{single,sg}. If you don't touch the data from the first pci_map_* | ||
575 | call till pci_unmap_*, then you don't have to call the pci_dma_sync_* | ||
576 | routines at all. | ||
577 | |||
578 | Here is pseudo code which shows a situation in which you would need | ||
579 | to use the pci_dma_sync_*() interfaces. | ||
580 | |||
581 | my_card_setup_receive_buffer(struct my_card *cp, char *buffer, int len) | ||
582 | { | ||
583 | dma_addr_t mapping; | ||
584 | |||
585 | mapping = pci_map_single(cp->pdev, buffer, len, PCI_DMA_FROMDEVICE); | ||
586 | |||
587 | cp->rx_buf = buffer; | ||
588 | cp->rx_len = len; | ||
589 | cp->rx_dma = mapping; | ||
590 | |||
591 | give_rx_buf_to_card(cp); | ||
592 | } | ||
593 | |||
594 | ... | ||
595 | |||
596 | my_card_interrupt_handler(int irq, void *devid, struct pt_regs *regs) | ||
597 | { | ||
598 | struct my_card *cp = devid; | ||
599 | |||
600 | ... | ||
601 | if (read_card_status(cp) == RX_BUF_TRANSFERRED) { | ||
602 | struct my_card_header *hp; | ||
603 | |||
604 | /* Examine the header to see if we wish | ||
605 | * to accept the data. But synchronize | ||
606 | * the DMA transfer with the CPU first | ||
607 | * so that we see updated contents. | ||
608 | */ | ||
609 | pci_dma_sync_single_for_cpu(cp->pdev, cp->rx_dma, | ||
610 | cp->rx_len, | ||
611 | PCI_DMA_FROMDEVICE); | ||
612 | |||
613 | /* Now it is safe to examine the buffer. */ | ||
614 | hp = (struct my_card_header *) cp->rx_buf; | ||
615 | if (header_is_ok(hp)) { | ||
616 | pci_unmap_single(cp->pdev, cp->rx_dma, cp->rx_len, | ||
617 | PCI_DMA_FROMDEVICE); | ||
618 | pass_to_upper_layers(cp->rx_buf); | ||
619 | make_and_setup_new_rx_buf(cp); | ||
620 | } else { | ||
621 | /* Just sync the buffer and give it back | ||
622 | * to the card. | ||
623 | */ | ||
624 | pci_dma_sync_single_for_device(cp->pdev, | ||
625 | cp->rx_dma, | ||
626 | cp->rx_len, | ||
627 | PCI_DMA_FROMDEVICE); | ||
628 | give_rx_buf_to_card(cp); | ||
629 | } | ||
630 | } | ||
631 | } | ||
632 | |||
633 | Drivers converted fully to this interface should not use virt_to_bus any | ||
634 | longer, nor should they use bus_to_virt. Some drivers have to be changed a | ||
635 | little bit, because there is no longer an equivalent to bus_to_virt in the | ||
636 | dynamic DMA mapping scheme - you have to always store the DMA addresses | ||
637 | returned by the pci_alloc_consistent, pci_pool_alloc, and pci_map_single | ||
638 | calls (pci_map_sg stores them in the scatterlist itself if the platform | ||
639 | supports dynamic DMA mapping in hardware) in your driver structures and/or | ||
640 | in the card registers. | ||
641 | |||
642 | All PCI drivers should be using these interfaces with no exceptions. | ||
643 | It is planned to completely remove virt_to_bus() and bus_to_virt() as | ||
644 | they are entirely deprecated. Some ports already do not provide these | ||
645 | as it is impossible to correctly support them. | ||
646 | |||
647 | 64-bit DMA and DAC cycle support | ||
648 | |||
649 | Do you understand all of the text above? Great, then you already | ||
650 | know how to use 64-bit DMA addressing under Linux. Simply make | ||
651 | the appropriate pci_set_dma_mask() calls based upon your cards | ||
652 | capabilities, then use the mapping APIs above. | ||
653 | |||
654 | It is that simple. | ||
655 | |||
656 | Well, not for some odd devices. See the next section for information | ||
657 | about that. | ||
658 | |||
659 | DAC Addressing for Address Space Hungry Devices | ||
660 | |||
661 | There exists a class of devices which do not mesh well with the PCI | ||
662 | DMA mapping API. By definition these "mappings" are a finite | ||
663 | resource. The number of total available mappings per bus is platform | ||
664 | specific, but there will always be a reasonable amount. | ||
665 | |||
666 | What is "reasonable"? Reasonable means that networking and block I/O | ||
667 | devices need not worry about using too many mappings. | ||
668 | |||
669 | As an example of a problematic device, consider compute cluster cards. | ||
670 | They can potentially need to access gigabytes of memory at once via | ||
671 | DMA. Dynamic mappings are unsuitable for this kind of access pattern. | ||
672 | |||
673 | To this end we've provided a small API by which a device driver | ||
674 | may use DAC cycles to directly address all of physical memory. | ||
675 | Not all platforms support this, but most do. It is easy to determine | ||
676 | whether the platform will work properly at probe time. | ||
677 | |||
678 | First, understand that there may be a SEVERE performance penalty for | ||
679 | using these interfaces on some platforms. Therefore, you MUST only | ||
680 | use these interfaces if it is absolutely required. %99 of devices can | ||
681 | use the normal APIs without any problems. | ||
682 | |||
683 | Note that for streaming type mappings you must either use these | ||
684 | interfaces, or the dynamic mapping interfaces above. You may not mix | ||
685 | usage of both for the same device. Such an act is illegal and is | ||
686 | guaranteed to put a banana in your tailpipe. | ||
687 | |||
688 | However, consistent mappings may in fact be used in conjunction with | ||
689 | these interfaces. Remember that, as defined, consistent mappings are | ||
690 | always going to be SAC addressable. | ||
691 | |||
692 | The first thing your driver needs to do is query the PCI platform | ||
693 | layer with your devices DAC addressing capabilities: | ||
694 | |||
695 | int pci_dac_set_dma_mask(struct pci_dev *pdev, u64 mask); | ||
696 | |||
697 | This routine behaves identically to pci_set_dma_mask. You may not | ||
698 | use the following interfaces if this routine fails. | ||
699 | |||
700 | Next, DMA addresses using this API are kept track of using the | ||
701 | dma64_addr_t type. It is guaranteed to be big enough to hold any | ||
702 | DAC address the platform layer will give to you from the following | ||
703 | routines. If you have consistent mappings as well, you still | ||
704 | use plain dma_addr_t to keep track of those. | ||
705 | |||
706 | All mappings obtained here will be direct. The mappings are not | ||
707 | translated, and this is the purpose of this dialect of the DMA API. | ||
708 | |||
709 | All routines work with page/offset pairs. This is the _ONLY_ way to | ||
710 | portably refer to any piece of memory. If you have a cpu pointer | ||
711 | (which may be validly DMA'd too) you may easily obtain the page | ||
712 | and offset using something like this: | ||
713 | |||
714 | struct page *page = virt_to_page(ptr); | ||
715 | unsigned long offset = offset_in_page(ptr); | ||
716 | |||
717 | Here are the interfaces: | ||
718 | |||
719 | dma64_addr_t pci_dac_page_to_dma(struct pci_dev *pdev, | ||
720 | struct page *page, | ||
721 | unsigned long offset, | ||
722 | int direction); | ||
723 | |||
724 | The DAC address for the tuple PAGE/OFFSET are returned. The direction | ||
725 | argument is the same as for pci_{map,unmap}_single(). The same rules | ||
726 | for cpu/device access apply here as for the streaming mapping | ||
727 | interfaces. To reiterate: | ||
728 | |||
729 | The cpu may touch the buffer before pci_dac_page_to_dma. | ||
730 | The device may touch the buffer after pci_dac_page_to_dma | ||
731 | is made, but the cpu may NOT. | ||
732 | |||
733 | When the DMA transfer is complete, invoke: | ||
734 | |||
735 | void pci_dac_dma_sync_single_for_cpu(struct pci_dev *pdev, | ||
736 | dma64_addr_t dma_addr, | ||
737 | size_t len, int direction); | ||
738 | |||
739 | This must be done before the CPU looks at the buffer again. | ||
740 | This interface behaves identically to pci_dma_sync_{single,sg}_for_cpu(). | ||
741 | |||
742 | And likewise, if you wish to let the device get back at the buffer after | ||
743 | the cpu has read/written it, invoke: | ||
744 | |||
745 | void pci_dac_dma_sync_single_for_device(struct pci_dev *pdev, | ||
746 | dma64_addr_t dma_addr, | ||
747 | size_t len, int direction); | ||
748 | |||
749 | before letting the device access the DMA area again. | ||
750 | |||
751 | If you need to get back to the PAGE/OFFSET tuple from a dma64_addr_t | ||
752 | the following interfaces are provided: | ||
753 | |||
754 | struct page *pci_dac_dma_to_page(struct pci_dev *pdev, | ||
755 | dma64_addr_t dma_addr); | ||
756 | unsigned long pci_dac_dma_to_offset(struct pci_dev *pdev, | ||
757 | dma64_addr_t dma_addr); | ||
758 | |||
759 | This is possible with the DAC interfaces purely because they are | ||
760 | not translated in any way. | ||
761 | |||
762 | Optimizing Unmap State Space Consumption | ||
763 | |||
764 | On many platforms, pci_unmap_{single,page}() is simply a nop. | ||
765 | Therefore, keeping track of the mapping address and length is a waste | ||
766 | of space. Instead of filling your drivers up with ifdefs and the like | ||
767 | to "work around" this (which would defeat the whole purpose of a | ||
768 | portable API) the following facilities are provided. | ||
769 | |||
770 | Actually, instead of describing the macros one by one, we'll | ||
771 | transform some example code. | ||
772 | |||
773 | 1) Use DECLARE_PCI_UNMAP_{ADDR,LEN} in state saving structures. | ||
774 | Example, before: | ||
775 | |||
776 | struct ring_state { | ||
777 | struct sk_buff *skb; | ||
778 | dma_addr_t mapping; | ||
779 | __u32 len; | ||
780 | }; | ||
781 | |||
782 | after: | ||
783 | |||
784 | struct ring_state { | ||
785 | struct sk_buff *skb; | ||
786 | DECLARE_PCI_UNMAP_ADDR(mapping) | ||
787 | DECLARE_PCI_UNMAP_LEN(len) | ||
788 | }; | ||
789 | |||
790 | NOTE: DO NOT put a semicolon at the end of the DECLARE_*() | ||
791 | macro. | ||
792 | |||
793 | 2) Use pci_unmap_{addr,len}_set to set these values. | ||
794 | Example, before: | ||
795 | |||
796 | ringp->mapping = FOO; | ||
797 | ringp->len = BAR; | ||
798 | |||
799 | after: | ||
800 | |||
801 | pci_unmap_addr_set(ringp, mapping, FOO); | ||
802 | pci_unmap_len_set(ringp, len, BAR); | ||
803 | |||
804 | 3) Use pci_unmap_{addr,len} to access these values. | ||
805 | Example, before: | ||
806 | |||
807 | pci_unmap_single(pdev, ringp->mapping, ringp->len, | ||
808 | PCI_DMA_FROMDEVICE); | ||
809 | |||
810 | after: | ||
811 | |||
812 | pci_unmap_single(pdev, | ||
813 | pci_unmap_addr(ringp, mapping), | ||
814 | pci_unmap_len(ringp, len), | ||
815 | PCI_DMA_FROMDEVICE); | ||
816 | |||
817 | It really should be self-explanatory. We treat the ADDR and LEN | ||
818 | separately, because it is possible for an implementation to only | ||
819 | need the address in order to perform the unmap operation. | ||
820 | |||
821 | Platform Issues | ||
822 | |||
823 | If you are just writing drivers for Linux and do not maintain | ||
824 | an architecture port for the kernel, you can safely skip down | ||
825 | to "Closing". | ||
826 | |||
827 | 1) Struct scatterlist requirements. | ||
828 | |||
829 | Struct scatterlist must contain, at a minimum, the following | ||
830 | members: | ||
831 | |||
832 | struct page *page; | ||
833 | unsigned int offset; | ||
834 | unsigned int length; | ||
835 | |||
836 | The base address is specified by a "page+offset" pair. | ||
837 | |||
838 | Previous versions of struct scatterlist contained a "void *address" | ||
839 | field that was sometimes used instead of page+offset. As of Linux | ||
840 | 2.5., page+offset is always used, and the "address" field has been | ||
841 | deleted. | ||
842 | |||
843 | 2) More to come... | ||
844 | |||
845 | Handling Errors | ||
846 | |||
847 | DMA address space is limited on some architectures and an allocation | ||
848 | failure can be determined by: | ||
849 | |||
850 | - checking if pci_alloc_consistent returns NULL or pci_map_sg returns 0 | ||
851 | |||
852 | - checking the returned dma_addr_t of pci_map_single and pci_map_page | ||
853 | by using pci_dma_mapping_error(): | ||
854 | |||
855 | dma_addr_t dma_handle; | ||
856 | |||
857 | dma_handle = pci_map_single(dev, addr, size, direction); | ||
858 | if (pci_dma_mapping_error(dma_handle)) { | ||
859 | /* | ||
860 | * reduce current DMA mapping usage, | ||
861 | * delay and try again later or | ||
862 | * reset driver. | ||
863 | */ | ||
864 | } | ||
865 | |||
866 | Closing | ||
867 | |||
868 | This document, and the API itself, would not be in it's current | ||
869 | form without the feedback and suggestions from numerous individuals. | ||
870 | We would like to specifically mention, in no particular order, the | ||
871 | following people: | ||
872 | |||
873 | Russell King <rmk@arm.linux.org.uk> | ||
874 | Leo Dagum <dagum@barrel.engr.sgi.com> | ||
875 | Ralf Baechle <ralf@oss.sgi.com> | ||
876 | Grant Grundler <grundler@cup.hp.com> | ||
877 | Jay Estabrook <Jay.Estabrook@compaq.com> | ||
878 | Thomas Sailer <sailer@ife.ee.ethz.ch> | ||
879 | Andrea Arcangeli <andrea@suse.de> | ||
880 | Jens Axboe <axboe@suse.de> | ||
881 | David Mosberger-Tang <davidm@hpl.hp.com> | ||