diff options
author | Peter Zijlstra <peterz@infradead.org> | 2013-11-06 08:57:36 -0500 |
---|---|---|
committer | Ingo Molnar <mingo@kernel.org> | 2014-01-12 04:37:13 -0500 |
commit | 2e4f5382d12a441b5cccfdde00308df15c2ce300 (patch) | |
tree | 7616367bf8021cded6e7635c2be3a360a995b101 /Documentation/memory-barriers.txt | |
parent | 91f30a17024ff0d8345e11228af33ee938b13426 (diff) |
locking/doc: Rename LOCK/UNLOCK to ACQUIRE/RELEASE
The LOCK and UNLOCK barriers as described in our barrier document are
generally known as ACQUIRE and RELEASE barriers in other literature.
Since we plan to introduce the acquire and release nomenclature in
generic kernel primitives we should amend the document to avoid
confusion as to what an acquire/release means.
Reviewed-by: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Acked-by: Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Michael Ellerman <michael@ellerman.id.au>
Cc: Michael Neuling <mikey@neuling.org>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Victor Kaplansky <VICTORK@il.ibm.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Oleg Nesterov <oleg@redhat.com>
Link: http://lkml.kernel.org/r/20131217092435.GC21999@twins.programming.kicks-ass.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Diffstat (limited to 'Documentation/memory-barriers.txt')
-rw-r--r-- | Documentation/memory-barriers.txt | 237 |
1 files changed, 121 insertions, 116 deletions
diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index cb753c8158f2..102dc19c4119 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt | |||
@@ -381,39 +381,44 @@ Memory barriers come in four basic varieties: | |||
381 | 381 | ||
382 | And a couple of implicit varieties: | 382 | And a couple of implicit varieties: |
383 | 383 | ||
384 | (5) LOCK operations. | 384 | (5) ACQUIRE operations. |
385 | 385 | ||
386 | This acts as a one-way permeable barrier. It guarantees that all memory | 386 | This acts as a one-way permeable barrier. It guarantees that all memory |
387 | operations after the LOCK operation will appear to happen after the LOCK | 387 | operations after the ACQUIRE operation will appear to happen after the |
388 | operation with respect to the other components of the system. | 388 | ACQUIRE operation with respect to the other components of the system. |
389 | ACQUIRE operations include LOCK operations and smp_load_acquire() | ||
390 | operations. | ||
389 | 391 | ||
390 | Memory operations that occur before a LOCK operation may appear to happen | 392 | Memory operations that occur before an ACQUIRE operation may appear to |
391 | after it completes. | 393 | happen after it completes. |
392 | 394 | ||
393 | A LOCK operation should almost always be paired with an UNLOCK operation. | 395 | An ACQUIRE operation should almost always be paired with a RELEASE |
396 | operation. | ||
394 | 397 | ||
395 | 398 | ||
396 | (6) UNLOCK operations. | 399 | (6) RELEASE operations. |
397 | 400 | ||
398 | This also acts as a one-way permeable barrier. It guarantees that all | 401 | This also acts as a one-way permeable barrier. It guarantees that all |
399 | memory operations before the UNLOCK operation will appear to happen before | 402 | memory operations before the RELEASE operation will appear to happen |
400 | the UNLOCK operation with respect to the other components of the system. | 403 | before the RELEASE operation with respect to the other components of the |
404 | system. RELEASE operations include UNLOCK operations and | ||
405 | smp_store_release() operations. | ||
401 | 406 | ||
402 | Memory operations that occur after an UNLOCK operation may appear to | 407 | Memory operations that occur after a RELEASE operation may appear to |
403 | happen before it completes. | 408 | happen before it completes. |
404 | 409 | ||
405 | The use of LOCK and UNLOCK operations generally precludes the need for | 410 | The use of ACQUIRE and RELEASE operations generally precludes the need |
406 | other sorts of memory barrier (but note the exceptions mentioned in the | 411 | for other sorts of memory barrier (but note the exceptions mentioned in |
407 | subsection "MMIO write barrier"). In addition, an UNLOCK+LOCK pair | 412 | the subsection "MMIO write barrier"). In addition, a RELEASE+ACQUIRE |
408 | is -not- guaranteed to act as a full memory barrier. However, | 413 | pair is -not- guaranteed to act as a full memory barrier. However, after |
409 | after a LOCK on a given lock variable, all memory accesses preceding any | 414 | an ACQUIRE on a given variable, all memory accesses preceding any prior |
410 | prior UNLOCK on that same variable are guaranteed to be visible. | 415 | RELEASE on that same variable are guaranteed to be visible. In other |
411 | In other words, within a given lock variable's critical section, | 416 | words, within a given variable's critical section, all accesses of all |
412 | all accesses of all previous critical sections for that lock variable | 417 | previous critical sections for that variable are guaranteed to have |
413 | are guaranteed to have completed. | 418 | completed. |
414 | 419 | ||
415 | This means that LOCK acts as a minimal "acquire" operation and | 420 | This means that ACQUIRE acts as a minimal "acquire" operation and |
416 | UNLOCK acts as a minimal "release" operation. | 421 | RELEASE acts as a minimal "release" operation. |
417 | 422 | ||
418 | 423 | ||
419 | Memory barriers are only required where there's a possibility of interaction | 424 | Memory barriers are only required where there's a possibility of interaction |
@@ -1585,7 +1590,7 @@ There are some more advanced barrier functions: | |||
1585 | clear_bit( ... ); | 1590 | clear_bit( ... ); |
1586 | 1591 | ||
1587 | This prevents memory operations before the clear leaking to after it. See | 1592 | This prevents memory operations before the clear leaking to after it. See |
1588 | the subsection on "Locking Functions" with reference to UNLOCK operation | 1593 | the subsection on "Locking Functions" with reference to RELEASE operation |
1589 | implications. | 1594 | implications. |
1590 | 1595 | ||
1591 | See Documentation/atomic_ops.txt for more information. See the "Atomic | 1596 | See Documentation/atomic_ops.txt for more information. See the "Atomic |
@@ -1619,8 +1624,8 @@ provide more substantial guarantees, but these may not be relied upon outside | |||
1619 | of arch specific code. | 1624 | of arch specific code. |
1620 | 1625 | ||
1621 | 1626 | ||
1622 | LOCKING FUNCTIONS | 1627 | ACQUIRING FUNCTIONS |
1623 | ----------------- | 1628 | ------------------- |
1624 | 1629 | ||
1625 | The Linux kernel has a number of locking constructs: | 1630 | The Linux kernel has a number of locking constructs: |
1626 | 1631 | ||
@@ -1631,106 +1636,106 @@ The Linux kernel has a number of locking constructs: | |||
1631 | (*) R/W semaphores | 1636 | (*) R/W semaphores |
1632 | (*) RCU | 1637 | (*) RCU |
1633 | 1638 | ||
1634 | In all cases there are variants on "LOCK" operations and "UNLOCK" operations | 1639 | In all cases there are variants on "ACQUIRE" operations and "RELEASE" operations |
1635 | for each construct. These operations all imply certain barriers: | 1640 | for each construct. These operations all imply certain barriers: |
1636 | 1641 | ||
1637 | (1) LOCK operation implication: | 1642 | (1) ACQUIRE operation implication: |
1638 | 1643 | ||
1639 | Memory operations issued after the LOCK will be completed after the LOCK | 1644 | Memory operations issued after the ACQUIRE will be completed after the |
1640 | operation has completed. | 1645 | ACQUIRE operation has completed. |
1641 | 1646 | ||
1642 | Memory operations issued before the LOCK may be completed after the | 1647 | Memory operations issued before the ACQUIRE may be completed after the |
1643 | LOCK operation has completed. An smp_mb__before_spinlock(), combined | 1648 | ACQUIRE operation has completed. An smp_mb__before_spinlock(), combined |
1644 | with a following LOCK, orders prior loads against subsequent stores | 1649 | with a following ACQUIRE, orders prior loads against subsequent stores and |
1645 | and stores and prior stores against subsequent stores. Note that | 1650 | stores and prior stores against subsequent stores. Note that this is |
1646 | this is weaker than smp_mb()! The smp_mb__before_spinlock() | 1651 | weaker than smp_mb()! The smp_mb__before_spinlock() primitive is free on |
1647 | primitive is free on many architectures. | 1652 | many architectures. |
1648 | 1653 | ||
1649 | (2) UNLOCK operation implication: | 1654 | (2) RELEASE operation implication: |
1650 | 1655 | ||
1651 | Memory operations issued before the UNLOCK will be completed before the | 1656 | Memory operations issued before the RELEASE will be completed before the |
1652 | UNLOCK operation has completed. | 1657 | RELEASE operation has completed. |
1653 | 1658 | ||
1654 | Memory operations issued after the UNLOCK may be completed before the | 1659 | Memory operations issued after the RELEASE may be completed before the |
1655 | UNLOCK operation has completed. | 1660 | RELEASE operation has completed. |
1656 | 1661 | ||
1657 | (3) LOCK vs LOCK implication: | 1662 | (3) ACQUIRE vs ACQUIRE implication: |
1658 | 1663 | ||
1659 | All LOCK operations issued before another LOCK operation will be completed | 1664 | All ACQUIRE operations issued before another ACQUIRE operation will be |
1660 | before that LOCK operation. | 1665 | completed before that ACQUIRE operation. |
1661 | 1666 | ||
1662 | (4) LOCK vs UNLOCK implication: | 1667 | (4) ACQUIRE vs RELEASE implication: |
1663 | 1668 | ||
1664 | All LOCK operations issued before an UNLOCK operation will be completed | 1669 | All ACQUIRE operations issued before a RELEASE operation will be |
1665 | before the UNLOCK operation. | 1670 | completed before the RELEASE operation. |
1666 | 1671 | ||
1667 | (5) Failed conditional LOCK implication: | 1672 | (5) Failed conditional ACQUIRE implication: |
1668 | 1673 | ||
1669 | Certain variants of the LOCK operation may fail, either due to being | 1674 | Certain locking variants of the ACQUIRE operation may fail, either due to |
1670 | unable to get the lock immediately, or due to receiving an unblocked | 1675 | being unable to get the lock immediately, or due to receiving an unblocked |
1671 | signal whilst asleep waiting for the lock to become available. Failed | 1676 | signal whilst asleep waiting for the lock to become available. Failed |
1672 | locks do not imply any sort of barrier. | 1677 | locks do not imply any sort of barrier. |
1673 | 1678 | ||
1674 | [!] Note: one of the consequences of LOCKs and UNLOCKs being only one-way | 1679 | [!] Note: one of the consequences of lock ACQUIREs and RELEASEs being only |
1675 | barriers is that the effects of instructions outside of a critical section | 1680 | one-way barriers is that the effects of instructions outside of a critical |
1676 | may seep into the inside of the critical section. | 1681 | section may seep into the inside of the critical section. |
1677 | 1682 | ||
1678 | A LOCK followed by an UNLOCK may not be assumed to be full memory barrier | 1683 | An ACQUIRE followed by a RELEASE may not be assumed to be full memory barrier |
1679 | because it is possible for an access preceding the LOCK to happen after the | 1684 | because it is possible for an access preceding the ACQUIRE to happen after the |
1680 | LOCK, and an access following the UNLOCK to happen before the UNLOCK, and the | 1685 | ACQUIRE, and an access following the RELEASE to happen before the RELEASE, and |
1681 | two accesses can themselves then cross: | 1686 | the two accesses can themselves then cross: |
1682 | 1687 | ||
1683 | *A = a; | 1688 | *A = a; |
1684 | LOCK M | 1689 | ACQUIRE M |
1685 | UNLOCK M | 1690 | RELEASE M |
1686 | *B = b; | 1691 | *B = b; |
1687 | 1692 | ||
1688 | may occur as: | 1693 | may occur as: |
1689 | 1694 | ||
1690 | LOCK M, STORE *B, STORE *A, UNLOCK M | 1695 | ACQUIRE M, STORE *B, STORE *A, RELEASE M |
1691 | 1696 | ||
1692 | This same reordering can of course occur if the LOCK and UNLOCK are | 1697 | This same reordering can of course occur if the lock's ACQUIRE and RELEASE are |
1693 | to the same lock variable, but only from the perspective of another | 1698 | to the same lock variable, but only from the perspective of another CPU not |
1694 | CPU not holding that lock. | 1699 | holding that lock. |
1695 | 1700 | ||
1696 | In short, an UNLOCK followed by a LOCK may -not- be assumed to be a full | 1701 | In short, a RELEASE followed by an ACQUIRE may -not- be assumed to be a full |
1697 | memory barrier because it is possible for a preceding UNLOCK to pass a | 1702 | memory barrier because it is possible for a preceding RELEASE to pass a |
1698 | later LOCK from the viewpoint of the CPU, but not from the viewpoint | 1703 | later ACQUIRE from the viewpoint of the CPU, but not from the viewpoint |
1699 | of the compiler. Note that deadlocks cannot be introduced by this | 1704 | of the compiler. Note that deadlocks cannot be introduced by this |
1700 | interchange because if such a deadlock threatened, the UNLOCK would | 1705 | interchange because if such a deadlock threatened, the RELEASE would |
1701 | simply complete. | 1706 | simply complete. |
1702 | 1707 | ||
1703 | If it is necessary for an UNLOCK-LOCK pair to produce a full barrier, | 1708 | If it is necessary for a RELEASE-ACQUIRE pair to produce a full barrier, the |
1704 | the LOCK can be followed by an smp_mb__after_unlock_lock() invocation. | 1709 | ACQUIRE can be followed by an smp_mb__after_unlock_lock() invocation. This |
1705 | This will produce a full barrier if either (a) the UNLOCK and the LOCK | 1710 | will produce a full barrier if either (a) the RELEASE and the ACQUIRE are |
1706 | are executed by the same CPU or task, or (b) the UNLOCK and LOCK act | 1711 | executed by the same CPU or task, or (b) the RELEASE and ACQUIRE act on the |
1707 | on the same lock variable. The smp_mb__after_unlock_lock() primitive | 1712 | same variable. The smp_mb__after_unlock_lock() primitive is free on many |
1708 | is free on many architectures. Without smp_mb__after_unlock_lock(), | 1713 | architectures. Without smp_mb__after_unlock_lock(), the critical sections |
1709 | the critical sections corresponding to the UNLOCK and the LOCK can cross: | 1714 | corresponding to the RELEASE and the ACQUIRE can cross: |
1710 | 1715 | ||
1711 | *A = a; | 1716 | *A = a; |
1712 | UNLOCK M | 1717 | RELEASE M |
1713 | LOCK N | 1718 | ACQUIRE N |
1714 | *B = b; | 1719 | *B = b; |
1715 | 1720 | ||
1716 | could occur as: | 1721 | could occur as: |
1717 | 1722 | ||
1718 | LOCK N, STORE *B, STORE *A, UNLOCK M | 1723 | ACQUIRE N, STORE *B, STORE *A, RELEASE M |
1719 | 1724 | ||
1720 | With smp_mb__after_unlock_lock(), they cannot, so that: | 1725 | With smp_mb__after_unlock_lock(), they cannot, so that: |
1721 | 1726 | ||
1722 | *A = a; | 1727 | *A = a; |
1723 | UNLOCK M | 1728 | RELEASE M |
1724 | LOCK N | 1729 | ACQUIRE N |
1725 | smp_mb__after_unlock_lock(); | 1730 | smp_mb__after_unlock_lock(); |
1726 | *B = b; | 1731 | *B = b; |
1727 | 1732 | ||
1728 | will always occur as either of the following: | 1733 | will always occur as either of the following: |
1729 | 1734 | ||
1730 | STORE *A, UNLOCK, LOCK, STORE *B | 1735 | STORE *A, RELEASE, ACQUIRE, STORE *B |
1731 | STORE *A, LOCK, UNLOCK, STORE *B | 1736 | STORE *A, ACQUIRE, RELEASE, STORE *B |
1732 | 1737 | ||
1733 | If the UNLOCK and LOCK were instead both operating on the same lock | 1738 | If the RELEASE and ACQUIRE were instead both operating on the same lock |
1734 | variable, only the first of these two alternatives can occur. | 1739 | variable, only the first of these two alternatives can occur. |
1735 | 1740 | ||
1736 | Locks and semaphores may not provide any guarantee of ordering on UP compiled | 1741 | Locks and semaphores may not provide any guarantee of ordering on UP compiled |
@@ -1745,33 +1750,33 @@ As an example, consider the following: | |||
1745 | 1750 | ||
1746 | *A = a; | 1751 | *A = a; |
1747 | *B = b; | 1752 | *B = b; |
1748 | LOCK | 1753 | ACQUIRE |
1749 | *C = c; | 1754 | *C = c; |
1750 | *D = d; | 1755 | *D = d; |
1751 | UNLOCK | 1756 | RELEASE |
1752 | *E = e; | 1757 | *E = e; |
1753 | *F = f; | 1758 | *F = f; |
1754 | 1759 | ||
1755 | The following sequence of events is acceptable: | 1760 | The following sequence of events is acceptable: |
1756 | 1761 | ||
1757 | LOCK, {*F,*A}, *E, {*C,*D}, *B, UNLOCK | 1762 | ACQUIRE, {*F,*A}, *E, {*C,*D}, *B, RELEASE |
1758 | 1763 | ||
1759 | [+] Note that {*F,*A} indicates a combined access. | 1764 | [+] Note that {*F,*A} indicates a combined access. |
1760 | 1765 | ||
1761 | But none of the following are: | 1766 | But none of the following are: |
1762 | 1767 | ||
1763 | {*F,*A}, *B, LOCK, *C, *D, UNLOCK, *E | 1768 | {*F,*A}, *B, ACQUIRE, *C, *D, RELEASE, *E |
1764 | *A, *B, *C, LOCK, *D, UNLOCK, *E, *F | 1769 | *A, *B, *C, ACQUIRE, *D, RELEASE, *E, *F |
1765 | *A, *B, LOCK, *C, UNLOCK, *D, *E, *F | 1770 | *A, *B, ACQUIRE, *C, RELEASE, *D, *E, *F |
1766 | *B, LOCK, *C, *D, UNLOCK, {*F,*A}, *E | 1771 | *B, ACQUIRE, *C, *D, RELEASE, {*F,*A}, *E |
1767 | 1772 | ||
1768 | 1773 | ||
1769 | 1774 | ||
1770 | INTERRUPT DISABLING FUNCTIONS | 1775 | INTERRUPT DISABLING FUNCTIONS |
1771 | ----------------------------- | 1776 | ----------------------------- |
1772 | 1777 | ||
1773 | Functions that disable interrupts (LOCK equivalent) and enable interrupts | 1778 | Functions that disable interrupts (ACQUIRE equivalent) and enable interrupts |
1774 | (UNLOCK equivalent) will act as compiler barriers only. So if memory or I/O | 1779 | (RELEASE equivalent) will act as compiler barriers only. So if memory or I/O |
1775 | barriers are required in such a situation, they must be provided from some | 1780 | barriers are required in such a situation, they must be provided from some |
1776 | other means. | 1781 | other means. |
1777 | 1782 | ||
@@ -1910,17 +1915,17 @@ Other functions that imply barriers: | |||
1910 | (*) schedule() and similar imply full memory barriers. | 1915 | (*) schedule() and similar imply full memory barriers. |
1911 | 1916 | ||
1912 | 1917 | ||
1913 | ================================= | 1918 | =================================== |
1914 | INTER-CPU LOCKING BARRIER EFFECTS | 1919 | INTER-CPU ACQUIRING BARRIER EFFECTS |
1915 | ================================= | 1920 | =================================== |
1916 | 1921 | ||
1917 | On SMP systems locking primitives give a more substantial form of barrier: one | 1922 | On SMP systems locking primitives give a more substantial form of barrier: one |
1918 | that does affect memory access ordering on other CPUs, within the context of | 1923 | that does affect memory access ordering on other CPUs, within the context of |
1919 | conflict on any particular lock. | 1924 | conflict on any particular lock. |
1920 | 1925 | ||
1921 | 1926 | ||
1922 | LOCKS VS MEMORY ACCESSES | 1927 | ACQUIRES VS MEMORY ACCESSES |
1923 | ------------------------ | 1928 | --------------------------- |
1924 | 1929 | ||
1925 | Consider the following: the system has a pair of spinlocks (M) and (Q), and | 1930 | Consider the following: the system has a pair of spinlocks (M) and (Q), and |
1926 | three CPUs; then should the following sequence of events occur: | 1931 | three CPUs; then should the following sequence of events occur: |
@@ -1928,24 +1933,24 @@ three CPUs; then should the following sequence of events occur: | |||
1928 | CPU 1 CPU 2 | 1933 | CPU 1 CPU 2 |
1929 | =============================== =============================== | 1934 | =============================== =============================== |
1930 | ACCESS_ONCE(*A) = a; ACCESS_ONCE(*E) = e; | 1935 | ACCESS_ONCE(*A) = a; ACCESS_ONCE(*E) = e; |
1931 | LOCK M LOCK Q | 1936 | ACQUIRE M ACQUIRE Q |
1932 | ACCESS_ONCE(*B) = b; ACCESS_ONCE(*F) = f; | 1937 | ACCESS_ONCE(*B) = b; ACCESS_ONCE(*F) = f; |
1933 | ACCESS_ONCE(*C) = c; ACCESS_ONCE(*G) = g; | 1938 | ACCESS_ONCE(*C) = c; ACCESS_ONCE(*G) = g; |
1934 | UNLOCK M UNLOCK Q | 1939 | RELEASE M RELEASE Q |
1935 | ACCESS_ONCE(*D) = d; ACCESS_ONCE(*H) = h; | 1940 | ACCESS_ONCE(*D) = d; ACCESS_ONCE(*H) = h; |
1936 | 1941 | ||
1937 | Then there is no guarantee as to what order CPU 3 will see the accesses to *A | 1942 | Then there is no guarantee as to what order CPU 3 will see the accesses to *A |
1938 | through *H occur in, other than the constraints imposed by the separate locks | 1943 | through *H occur in, other than the constraints imposed by the separate locks |
1939 | on the separate CPUs. It might, for example, see: | 1944 | on the separate CPUs. It might, for example, see: |
1940 | 1945 | ||
1941 | *E, LOCK M, LOCK Q, *G, *C, *F, *A, *B, UNLOCK Q, *D, *H, UNLOCK M | 1946 | *E, ACQUIRE M, ACQUIRE Q, *G, *C, *F, *A, *B, RELEASE Q, *D, *H, RELEASE M |
1942 | 1947 | ||
1943 | But it won't see any of: | 1948 | But it won't see any of: |
1944 | 1949 | ||
1945 | *B, *C or *D preceding LOCK M | 1950 | *B, *C or *D preceding ACQUIRE M |
1946 | *A, *B or *C following UNLOCK M | 1951 | *A, *B or *C following RELEASE M |
1947 | *F, *G or *H preceding LOCK Q | 1952 | *F, *G or *H preceding ACQUIRE Q |
1948 | *E, *F or *G following UNLOCK Q | 1953 | *E, *F or *G following RELEASE Q |
1949 | 1954 | ||
1950 | 1955 | ||
1951 | However, if the following occurs: | 1956 | However, if the following occurs: |
@@ -1953,29 +1958,29 @@ However, if the following occurs: | |||
1953 | CPU 1 CPU 2 | 1958 | CPU 1 CPU 2 |
1954 | =============================== =============================== | 1959 | =============================== =============================== |
1955 | ACCESS_ONCE(*A) = a; | 1960 | ACCESS_ONCE(*A) = a; |
1956 | LOCK M [1] | 1961 | ACQUIRE M [1] |
1957 | ACCESS_ONCE(*B) = b; | 1962 | ACCESS_ONCE(*B) = b; |
1958 | ACCESS_ONCE(*C) = c; | 1963 | ACCESS_ONCE(*C) = c; |
1959 | UNLOCK M [1] | 1964 | RELEASE M [1] |
1960 | ACCESS_ONCE(*D) = d; ACCESS_ONCE(*E) = e; | 1965 | ACCESS_ONCE(*D) = d; ACCESS_ONCE(*E) = e; |
1961 | LOCK M [2] | 1966 | ACQUIRE M [2] |
1962 | smp_mb__after_unlock_lock(); | 1967 | smp_mb__after_unlock_lock(); |
1963 | ACCESS_ONCE(*F) = f; | 1968 | ACCESS_ONCE(*F) = f; |
1964 | ACCESS_ONCE(*G) = g; | 1969 | ACCESS_ONCE(*G) = g; |
1965 | UNLOCK M [2] | 1970 | RELEASE M [2] |
1966 | ACCESS_ONCE(*H) = h; | 1971 | ACCESS_ONCE(*H) = h; |
1967 | 1972 | ||
1968 | CPU 3 might see: | 1973 | CPU 3 might see: |
1969 | 1974 | ||
1970 | *E, LOCK M [1], *C, *B, *A, UNLOCK M [1], | 1975 | *E, ACQUIRE M [1], *C, *B, *A, RELEASE M [1], |
1971 | LOCK M [2], *H, *F, *G, UNLOCK M [2], *D | 1976 | ACQUIRE M [2], *H, *F, *G, RELEASE M [2], *D |
1972 | 1977 | ||
1973 | But assuming CPU 1 gets the lock first, CPU 3 won't see any of: | 1978 | But assuming CPU 1 gets the lock first, CPU 3 won't see any of: |
1974 | 1979 | ||
1975 | *B, *C, *D, *F, *G or *H preceding LOCK M [1] | 1980 | *B, *C, *D, *F, *G or *H preceding ACQUIRE M [1] |
1976 | *A, *B or *C following UNLOCK M [1] | 1981 | *A, *B or *C following RELEASE M [1] |
1977 | *F, *G or *H preceding LOCK M [2] | 1982 | *F, *G or *H preceding ACQUIRE M [2] |
1978 | *A, *B, *C, *E, *F or *G following UNLOCK M [2] | 1983 | *A, *B, *C, *E, *F or *G following RELEASE M [2] |
1979 | 1984 | ||
1980 | Note that the smp_mb__after_unlock_lock() is critically important | 1985 | Note that the smp_mb__after_unlock_lock() is critically important |
1981 | here: Without it CPU 3 might see some of the above orderings. | 1986 | here: Without it CPU 3 might see some of the above orderings. |
@@ -1983,8 +1988,8 @@ Without smp_mb__after_unlock_lock(), the accesses are not guaranteed | |||
1983 | to be seen in order unless CPU 3 holds lock M. | 1988 | to be seen in order unless CPU 3 holds lock M. |
1984 | 1989 | ||
1985 | 1990 | ||
1986 | LOCKS VS I/O ACCESSES | 1991 | ACQUIRES VS I/O ACCESSES |
1987 | --------------------- | 1992 | ------------------------ |
1988 | 1993 | ||
1989 | Under certain circumstances (especially involving NUMA), I/O accesses within | 1994 | Under certain circumstances (especially involving NUMA), I/O accesses within |
1990 | two spinlocked sections on two different CPUs may be seen as interleaved by the | 1995 | two spinlocked sections on two different CPUs may be seen as interleaved by the |
@@ -2202,13 +2207,13 @@ explicit lock operations, described later). These include: | |||
2202 | /* when succeeds (returns 1) */ | 2207 | /* when succeeds (returns 1) */ |
2203 | atomic_add_unless(); atomic_long_add_unless(); | 2208 | atomic_add_unless(); atomic_long_add_unless(); |
2204 | 2209 | ||
2205 | These are used for such things as implementing LOCK-class and UNLOCK-class | 2210 | These are used for such things as implementing ACQUIRE-class and RELEASE-class |
2206 | operations and adjusting reference counters towards object destruction, and as | 2211 | operations and adjusting reference counters towards object destruction, and as |
2207 | such the implicit memory barrier effects are necessary. | 2212 | such the implicit memory barrier effects are necessary. |
2208 | 2213 | ||
2209 | 2214 | ||
2210 | The following operations are potential problems as they do _not_ imply memory | 2215 | The following operations are potential problems as they do _not_ imply memory |
2211 | barriers, but might be used for implementing such things as UNLOCK-class | 2216 | barriers, but might be used for implementing such things as RELEASE-class |
2212 | operations: | 2217 | operations: |
2213 | 2218 | ||
2214 | atomic_set(); | 2219 | atomic_set(); |
@@ -2250,7 +2255,7 @@ The following operations are special locking primitives: | |||
2250 | clear_bit_unlock(); | 2255 | clear_bit_unlock(); |
2251 | __clear_bit_unlock(); | 2256 | __clear_bit_unlock(); |
2252 | 2257 | ||
2253 | These implement LOCK-class and UNLOCK-class operations. These should be used in | 2258 | These implement ACQUIRE-class and RELEASE-class operations. These should be used in |
2254 | preference to other operations when implementing locking primitives, because | 2259 | preference to other operations when implementing locking primitives, because |
2255 | their implementations can be optimised on many architectures. | 2260 | their implementations can be optimised on many architectures. |
2256 | 2261 | ||