diff options
Diffstat (limited to 'Documentation/infiniband')
-rw-r--r-- | Documentation/infiniband/core_locking.txt | 114 |
1 files changed, 114 insertions, 0 deletions
diff --git a/Documentation/infiniband/core_locking.txt b/Documentation/infiniband/core_locking.txt new file mode 100644 index 000000000000..e1678542279a --- /dev/null +++ b/Documentation/infiniband/core_locking.txt | |||
@@ -0,0 +1,114 @@ | |||
1 | INFINIBAND MIDLAYER LOCKING | ||
2 | |||
3 | This guide is an attempt to make explicit the locking assumptions | ||
4 | made by the InfiniBand midlayer. It describes the requirements on | ||
5 | both low-level drivers that sit below the midlayer and upper level | ||
6 | protocols that use the midlayer. | ||
7 | |||
8 | Sleeping and interrupt context | ||
9 | |||
10 | With the following exceptions, a low-level driver implementation of | ||
11 | all of the methods in struct ib_device may sleep. The exceptions | ||
12 | are any methods from the list: | ||
13 | |||
14 | create_ah | ||
15 | modify_ah | ||
16 | query_ah | ||
17 | destroy_ah | ||
18 | bind_mw | ||
19 | post_send | ||
20 | post_recv | ||
21 | poll_cq | ||
22 | req_notify_cq | ||
23 | map_phys_fmr | ||
24 | |||
25 | which may not sleep and must be callable from any context. | ||
26 | |||
27 | The corresponding functions exported to upper level protocol | ||
28 | consumers: | ||
29 | |||
30 | ib_create_ah | ||
31 | ib_modify_ah | ||
32 | ib_query_ah | ||
33 | ib_destroy_ah | ||
34 | ib_bind_mw | ||
35 | ib_post_send | ||
36 | ib_post_recv | ||
37 | ib_req_notify_cq | ||
38 | ib_map_phys_fmr | ||
39 | |||
40 | are therefore safe to call from any context. | ||
41 | |||
42 | In addition, the function | ||
43 | |||
44 | ib_dispatch_event | ||
45 | |||
46 | used by low-level drivers to dispatch asynchronous events through | ||
47 | the midlayer is also safe to call from any context. | ||
48 | |||
49 | Reentrancy | ||
50 | |||
51 | All of the methods in struct ib_device exported by a low-level | ||
52 | driver must be fully reentrant. The low-level driver is required to | ||
53 | perform all synchronization necessary to maintain consistency, even | ||
54 | if multiple function calls using the same object are run | ||
55 | simultaneously. | ||
56 | |||
57 | The IB midlayer does not perform any serialization of function calls. | ||
58 | |||
59 | Because low-level drivers are reentrant, upper level protocol | ||
60 | consumers are not required to perform any serialization. However, | ||
61 | some serialization may be required to get sensible results. For | ||
62 | example, a consumer may safely call ib_poll_cq() on multiple CPUs | ||
63 | simultaneously. However, the ordering of the work completion | ||
64 | information between different calls of ib_poll_cq() is not defined. | ||
65 | |||
66 | Callbacks | ||
67 | |||
68 | A low-level driver must not perform a callback directly from the | ||
69 | same callchain as an ib_device method call. For example, it is not | ||
70 | allowed for a low-level driver to call a consumer's completion event | ||
71 | handler directly from its post_send method. Instead, the low-level | ||
72 | driver should defer this callback by, for example, scheduling a | ||
73 | tasklet to perform the callback. | ||
74 | |||
75 | The low-level driver is responsible for ensuring that multiple | ||
76 | completion event handlers for the same CQ are not called | ||
77 | simultaneously. The driver must guarantee that only one CQ event | ||
78 | handler for a given CQ is running at a time. In other words, the | ||
79 | following situation is not allowed: | ||
80 | |||
81 | CPU1 CPU2 | ||
82 | |||
83 | low-level driver -> | ||
84 | consumer CQ event callback: | ||
85 | /* ... */ | ||
86 | ib_req_notify_cq(cq, ...); | ||
87 | low-level driver -> | ||
88 | /* ... */ consumer CQ event callback: | ||
89 | /* ... */ | ||
90 | return from CQ event handler | ||
91 | |||
92 | The context in which completion event and asynchronous event | ||
93 | callbacks run is not defined. Depending on the low-level driver, it | ||
94 | may be process context, softirq context, or interrupt context. | ||
95 | Upper level protocol consumers may not sleep in a callback. | ||
96 | |||
97 | Hot-plug | ||
98 | |||
99 | A low-level driver announces that a device is ready for use by | ||
100 | consumers when it calls ib_register_device(), all initialization | ||
101 | must be complete before this call. The device must remain usable | ||
102 | until the driver's call to ib_unregister_device() has returned. | ||
103 | |||
104 | A low-level driver must call ib_register_device() and | ||
105 | ib_unregister_device() from process context. It must not hold any | ||
106 | semaphores that could cause deadlock if a consumer calls back into | ||
107 | the driver across these calls. | ||
108 | |||
109 | An upper level protocol consumer may begin using an IB device as | ||
110 | soon as the add method of its struct ib_client is called for that | ||
111 | device. A consumer must finish all cleanup and free all resources | ||
112 | relating to a device before returning from the remove method. | ||
113 | |||
114 | A consumer is permitted to sleep in its add and remove methods. | ||