diff options
-rw-r--r-- | Documentation/memory-barriers.txt | 237 |
1 files changed, 121 insertions, 116 deletions
diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt index cb753c8158f2..102dc19c4119 100644 --- a/Documentation/memory-barriers.txt +++ b/Documentation/memory-barriers.txt | |||
@@ -381,39 +381,44 @@ Memory barriers come in four basic varieties: | |||
381 | 381 | ||
382 | And a couple of implicit varieties: | 382 | And a couple of implicit varieties: |
383 | 383 | ||
384 | (5) LOCK operations. | 384 | (5) ACQUIRE operations. |
385 | 385 | ||
386 | This acts as a one-way permeable barrier. It guarantees that all memory | 386 | This acts as a one-way permeable barrier. It guarantees that all memory |
387 | operations after the LOCK operation will appear to happen after the LOCK | 387 | operations after the ACQUIRE operation will appear to happen after the |
388 | operation with respect to the other components of the system. | 388 | ACQUIRE operation with respect to the other components of the system. |
389 | ACQUIRE operations include LOCK operations and smp_load_acquire() | ||
390 | operations. | ||
389 | 391 | ||
390 | Memory operations that occur before a LOCK operation may appear to happen | 392 | Memory operations that occur before an ACQUIRE operation may appear to |
391 | after it completes. | 393 | happen after it completes. |
392 | 394 | ||
393 | A LOCK operation should almost always be paired with an UNLOCK operation. | 395 | An ACQUIRE operation should almost always be paired with a RELEASE |
396 | operation. | ||
394 | 397 | ||
395 | 398 | ||
396 | (6) UNLOCK operations. | 399 | (6) RELEASE operations. |
397 | 400 | ||
398 | This also acts as a one-way permeable barrier. It guarantees that all | 401 | This also acts as a one-way permeable barrier. It guarantees that all |
399 | memory operations before the UNLOCK operation will appear to happen before | 402 | memory operations before the RELEASE operation will appear to happen |
400 | the UNLOCK operation with respect to the other components of the system. | 403 | before the RELEASE operation with respect to the other components of the |
404 | system. RELEASE operations include UNLOCK operations and | ||
405 | smp_store_release() operations. | ||
401 | 406 | ||
402 | Memory operations that occur after an UNLOCK operation may appear to | 407 | Memory operations that occur after a RELEASE operation may appear to |
403 | happen before it completes. | 408 | happen before it completes. |
404 | 409 | ||
405 | The use of LOCK and UNLOCK operations generally precludes the need for | 410 | The use of ACQUIRE and RELEASE operations generally precludes the need |
406 | other sorts of memory barrier (but note the exceptions mentioned in the | 411 | for other sorts of memory barrier (but note the exceptions mentioned in |
407 | subsection "MMIO write barrier"). In addition, an UNLOCK+LOCK pair | 412 | the subsection "MMIO write barrier"). In addition, a RELEASE+ACQUIRE |
408 | is -not- guaranteed to act as a full memory barrier. However, | 413 | pair is -not- guaranteed to act as a full memory barrier. However, after |
409 | after a LOCK on a given lock variable, all memory accesses preceding any | 414 | an ACQUIRE on a given variable, all memory accesses preceding any prior |
410 | prior UNLOCK on that same variable are guaranteed to be visible. | 415 | RELEASE on that same variable are guaranteed to be visible. In other |
411 | In other words, within a given lock variable's critical section, | 416 | words, within a given variable's critical section, all accesses of all |
412 | all accesses of all previous critical sections for that lock variable | 417 | previous critical sections for that variable are guaranteed to have |
413 | are guaranteed to have completed. | 418 | completed. |
414 | 419 | ||
415 | This means that LOCK acts as a minimal "acquire" operation and | 420 | This means that ACQUIRE acts as a minimal "acquire" operation and |
416 | UNLOCK acts as a minimal "release" operation. | 421 | RELEASE acts as a minimal "release" operation. |
417 | 422 | ||
418 | 423 | ||
419 | Memory barriers are only required where there's a possibility of interaction | 424 | Memory barriers are only required where there's a possibility of interaction |
@@ -1585,7 +1590,7 @@ There are some more advanced barrier functions: | |||
1585 | clear_bit( ... ); | 1590 | clear_bit( ... ); |
1586 | 1591 | ||
1587 | This prevents memory operations before the clear leaking to after it. See | 1592 | This prevents memory operations before the clear leaking to after it. See |
1588 | the subsection on "Locking Functions" with reference to UNLOCK operation | 1593 | the subsection on "Locking Functions" with reference to RELEASE operation |
1589 | implications. | 1594 | implications. |
1590 | 1595 | ||
1591 | See Documentation/atomic_ops.txt for more information. See the "Atomic | 1596 | See Documentation/atomic_ops.txt for more information. See the "Atomic |
@@ -1619,8 +1624,8 @@ provide more substantial guarantees, but these may not be relied upon outside | |||
1619 | of arch specific code. | 1624 | of arch specific code. |
1620 | 1625 | ||
1621 | 1626 | ||
1622 | LOCKING FUNCTIONS | 1627 | ACQUIRING FUNCTIONS |
1623 | ----------------- | 1628 | ------------------- |
1624 | 1629 | ||
1625 | The Linux kernel has a number of locking constructs: | 1630 | The Linux kernel has a number of locking constructs: |
1626 | 1631 | ||
@@ -1631,106 +1636,106 @@ The Linux kernel has a number of locking constructs: | |||
1631 | (*) R/W semaphores | 1636 | (*) R/W semaphores |
1632 | (*) RCU | 1637 | (*) RCU |
1633 | 1638 | ||
1634 | In all cases there are variants on "LOCK" operations and "UNLOCK" operations | 1639 | In all cases there are variants on "ACQUIRE" operations and "RELEASE" operations |
1635 | for each construct. These operations all imply certain barriers: | 1640 | for each construct. These operations all imply certain barriers: |
1636 | 1641 | ||
1637 | (1) LOCK operation implication: | 1642 | (1) ACQUIRE operation implication: |
1638 | 1643 | ||
1639 | Memory operations issued after the LOCK will be completed after the LOCK | 1644 | Memory operations issued after the ACQUIRE will be completed after the |
1640 | operation has completed. | 1645 | ACQUIRE operation has completed. |
1641 | 1646 | ||
1642 | Memory operations issued before the LOCK may be completed after the | 1647 | Memory operations issued before the ACQUIRE may be completed after the |
1643 | LOCK operation has completed. An smp_mb__before_spinlock(), combined | 1648 | ACQUIRE operation has completed. An smp_mb__before_spinlock(), combined |
1644 | with a following LOCK, orders prior loads against subsequent stores | 1649 | with a following ACQUIRE, orders prior loads against subsequent stores and |
1645 | and stores and prior stores against subsequent stores. Note that | 1650 | stores and prior stores against subsequent stores. Note that this is |
1646 | this is weaker than smp_mb()! The smp_mb__before_spinlock() | 1651 | weaker than smp_mb()! The smp_mb__before_spinlock() primitive is free on |
1647 | primitive is free on many architectures. | 1652 | many architectures. |
1648 | 1653 | ||
1649 | (2) UNLOCK operation implication: | 1654 | (2) RELEASE operation implication: |
1650 | 1655 | ||
1651 | Memory operations issued before the UNLOCK will be completed before the | 1656 | Memory operations issued before the RELEASE will be completed before the |
1652 | UNLOCK operation has completed. | 1657 | RELEASE operation has completed. |
1653 | 1658 | ||
1654 | Memory operations issued after the UNLOCK may be completed before the | 1659 | Memory operations issued after the RELEASE may be completed before the |
1655 | UNLOCK operation has completed. | 1660 | RELEASE operation has completed. |
1656 | 1661 | ||
1657 | (3) LOCK vs LOCK implication: | 1662 | (3) ACQUIRE vs ACQUIRE implication: |
1658 | 1663 | ||
1659 | All LOCK operations issued before another LOCK operation will be completed | 1664 | All ACQUIRE operations issued before another ACQUIRE operation will be |
1660 | before that LOCK operation. | 1665 | completed before that ACQUIRE operation. |
1661 | 1666 | ||
1662 | (4) LOCK vs UNLOCK implication: | 1667 | (4) ACQUIRE vs RELEASE implication: |
1663 | 1668 | ||
1664 | All LOCK operations issued before an UNLOCK operation will be completed | 1669 | All ACQUIRE operations issued before a RELEASE operation will be |
1665 | before the UNLOCK operation. | 1670 | completed before the RELEASE operation. |
1666 | 1671 | ||
1667 | (5) Failed conditional LOCK implication: | 1672 | (5) Failed conditional ACQUIRE implication: |
1668 | 1673 | ||
1669 | Certain variants of the LOCK operation may fail, either due to being | 1674 | Certain locking variants of the ACQUIRE operation may fail, either due to |
1670 | unable to get the lock immediately, or due to receiving an unblocked | 1675 | being unable to get the lock immediately, or due to receiving an unblocked |
1671 | signal whilst asleep waiting for the lock to become available. Failed | 1676 | signal whilst asleep waiting for the lock to become available. Failed |
1672 | locks do not imply any sort of barrier. | 1677 | locks do not imply any sort of barrier. |
1673 | 1678 | ||
1674 | [!] Note: one of the consequences of LOCKs and UNLOCKs being only one-way | 1679 | [!] Note: one of the consequences of lock ACQUIREs and RELEASEs being only |
1675 | barriers is that the effects of instructions outside of a critical section | 1680 | one-way barriers is that the effects of instructions outside of a critical |
1676 | may seep into the inside of the critical section. | 1681 | section may seep into the inside of the critical section. |
1677 | 1682 | ||
1678 | A LOCK followed by an UNLOCK may not be assumed to be full memory barrier | 1683 | An ACQUIRE followed by a RELEASE may not be assumed to be full memory barrier |
1679 | because it is possible for an access preceding the LOCK to happen after the | 1684 | because it is possible for an access preceding the ACQUIRE to happen after the |
1680 | LOCK, and an access following the UNLOCK to happen before the UNLOCK, and the | 1685 | ACQUIRE, and an access following the RELEASE to happen before the RELEASE, and |
1681 | two accesses can themselves then cross: | 1686 | the two accesses can themselves then cross: |
1682 | 1687 | ||
1683 | *A = a; | 1688 | *A = a; |
1684 | LOCK M | 1689 | ACQUIRE M |
1685 | UNLOCK M | 1690 | RELEASE M |
1686 | *B = b; | 1691 | *B = b; |
1687 | 1692 | ||
1688 | may occur as: | 1693 | may occur as: |
1689 | 1694 | ||
1690 | LOCK M, STORE *B, STORE *A, UNLOCK M | 1695 | ACQUIRE M, STORE *B, STORE *A, RELEASE M |
1691 | 1696 | ||
1692 | This same reordering can of course occur if the LOCK and UNLOCK are | 1697 | This same reordering can of course occur if the lock's ACQUIRE and RELEASE are |
1693 | to the same lock variable, but only from the perspective of another | 1698 | to the same lock variable, but only from the perspective of another CPU not |
1694 | CPU not holding that lock. | 1699 | holding that lock. |
1695 | 1700 | ||
1696 | In short, an UNLOCK followed by a LOCK may -not- be assumed to be a full | 1701 | In short, a RELEASE followed by an ACQUIRE may -not- be assumed to be a full |
1697 | memory barrier because it is possible for a preceding UNLOCK to pass a | 1702 | memory barrier because it is possible for a preceding RELEASE to pass a |
1698 | later LOCK from the viewpoint of the CPU, but not from the viewpoint | 1703 | later ACQUIRE from the viewpoint of the CPU, but not from the viewpoint |
1699 | of the compiler. Note that deadlocks cannot be introduced by this | 1704 | of the compiler. Note that deadlocks cannot be introduced by this |
1700 | interchange because if such a deadlock threatened, the UNLOCK would | 1705 | interchange because if such a deadlock threatened, the RELEASE would |
1701 | simply complete. | 1706 | simply complete. |
1702 | 1707 | ||
1703 | If it is necessary for an UNLOCK-LOCK pair to produce a full barrier, | 1708 | If it is necessary for a RELEASE-ACQUIRE pair to produce a full barrier, the |
1704 | the LOCK can be followed by an smp_mb__after_unlock_lock() invocation. | 1709 | ACQUIRE can be followed by an smp_mb__after_unlock_lock() invocation. This |
1705 | This will produce a full barrier if either (a) the UNLOCK and the LOCK | 1710 | will produce a full barrier if either (a) the RELEASE and the ACQUIRE are |
1706 | are executed by the same CPU or task, or (b) the UNLOCK and LOCK act | 1711 | executed by the same CPU or task, or (b) the RELEASE and ACQUIRE act on the |
1707 | on the same lock variable. The smp_mb__after_unlock_lock() primitive | 1712 | same variable. The smp_mb__after_unlock_lock() primitive is free on many |
1708 | is free on many architectures. Without smp_mb__after_unlock_lock(), | 1713 | architectures. Without smp_mb__after_unlock_lock(), the critical sections |
1709 | the critical sections corresponding to the UNLOCK and the LOCK can cross: | 1714 | corresponding to the RELEASE and the ACQUIRE can cross: |
1710 | 1715 | ||
1711 | *A = a; | 1716 | *A = a; |
1712 | UNLOCK M | 1717 | RELEASE M |
1713 | LOCK N | 1718 | ACQUIRE N |
1714 | *B = b; | 1719 | *B = b; |
1715 | 1720 | ||
1716 | could occur as: | 1721 | could occur as: |
1717 | 1722 | ||
1718 | LOCK N, STORE *B, STORE *A, UNLOCK M | 1723 | ACQUIRE N, STORE *B, STORE *A, RELEASE M |
1719 | 1724 | ||
1720 | With smp_mb__after_unlock_lock(), they cannot, so that: | 1725 | With smp_mb__after_unlock_lock(), they cannot, so that: |
1721 | 1726 | ||
1722 | *A = a; | 1727 | *A = a; |
1723 | UNLOCK M | 1728 | RELEASE M |
1724 | LOCK N | 1729 | ACQUIRE N |
1725 | smp_mb__after_unlock_lock(); | 1730 | smp_mb__after_unlock_lock(); |
1726 | *B = b; | 1731 | *B = b; |
1727 | 1732 | ||
1728 | will always occur as either of the following: | 1733 | will always occur as either of the following: |
1729 | 1734 | ||
1730 | STORE *A, UNLOCK, LOCK, STORE *B | 1735 | STORE *A, RELEASE, ACQUIRE, STORE *B |
1731 | STORE *A, LOCK, UNLOCK, STORE *B | 1736 | STORE *A, ACQUIRE, RELEASE, STORE *B |
1732 | 1737 | ||
1733 | If the UNLOCK and LOCK were instead both operating on the same lock | 1738 | If the RELEASE and ACQUIRE were instead both operating on the same lock |
1734 | variable, only the first of these two alternatives can occur. | 1739 | variable, only the first of these two alternatives can occur. |
1735 | 1740 | ||
1736 | Locks and semaphores may not provide any guarantee of ordering on UP compiled | 1741 | Locks and semaphores may not provide any guarantee of ordering on UP compiled |
@@ -1745,33 +1750,33 @@ As an example, consider the following: | |||
1745 | 1750 | ||
1746 | *A = a; | 1751 | *A = a; |
1747 | *B = b; | 1752 | *B = b; |
1748 | LOCK | 1753 | ACQUIRE |
1749 | *C = c; | 1754 | *C = c; |
1750 | *D = d; | 1755 | *D = d; |
1751 | UNLOCK | 1756 | RELEASE |
1752 | *E = e; | 1757 | *E = e; |
1753 | *F = f; | 1758 | *F = f; |
1754 | 1759 | ||
1755 | The following sequence of events is acceptable: | 1760 | The following sequence of events is acceptable: |
1756 | 1761 | ||
1757 | LOCK, {*F,*A}, *E, {*C,*D}, *B, UNLOCK | 1762 | ACQUIRE, {*F,*A}, *E, {*C,*D}, *B, RELEASE |
1758 | 1763 | ||
1759 | [+] Note that {*F,*A} indicates a combined access. | 1764 | [+] Note that {*F,*A} indicates a combined access. |
1760 | 1765 | ||
1761 | But none of the following are: | 1766 | But none of the following are: |
1762 | 1767 | ||
1763 | {*F,*A}, *B, LOCK, *C, *D, UNLOCK, *E | 1768 | {*F,*A}, *B, ACQUIRE, *C, *D, RELEASE, *E |
1764 | *A, *B, *C, LOCK, *D, UNLOCK, *E, *F | 1769 | *A, *B, *C, ACQUIRE, *D, RELEASE, *E, *F |
1765 | *A, *B, LOCK, *C, UNLOCK, *D, *E, *F | 1770 | *A, *B, ACQUIRE, *C, RELEASE, *D, *E, *F |
1766 | *B, LOCK, *C, *D, UNLOCK, {*F,*A}, *E | 1771 | *B, ACQUIRE, *C, *D, RELEASE, {*F,*A}, *E |
1767 | 1772 | ||
1768 | 1773 | ||
1769 | 1774 | ||
1770 | INTERRUPT DISABLING FUNCTIONS | 1775 | INTERRUPT DISABLING FUNCTIONS |
1771 | ----------------------------- | 1776 | ----------------------------- |
1772 | 1777 | ||
1773 | Functions that disable interrupts (LOCK equivalent) and enable interrupts | 1778 | Functions that disable interrupts (ACQUIRE equivalent) and enable interrupts |
1774 | (UNLOCK equivalent) will act as compiler barriers only. So if memory or I/O | 1779 | (RELEASE equivalent) will act as compiler barriers only. So if memory or I/O |
1775 | barriers are required in such a situation, they must be provided from some | 1780 | barriers are required in such a situation, they must be provided from some |
1776 | other means. | 1781 | other means. |
1777 | 1782 | ||
@@ -1910,17 +1915,17 @@ Other functions that imply barriers: | |||
1910 | (*) schedule() and similar imply full memory barriers. | 1915 | (*) schedule() and similar imply full memory barriers. |
1911 | 1916 | ||
1912 | 1917 | ||
1913 | ================================= | 1918 | =================================== |
1914 | INTER-CPU LOCKING BARRIER EFFECTS | 1919 | INTER-CPU ACQUIRING BARRIER EFFECTS |
1915 | ================================= | 1920 | =================================== |
1916 | 1921 | ||
1917 | On SMP systems locking primitives give a more substantial form of barrier: one | 1922 | On SMP systems locking primitives give a more substantial form of barrier: one |
1918 | that does affect memory access ordering on other CPUs, within the context of | 1923 | that does affect memory access ordering on other CPUs, within the context of |
1919 | conflict on any particular lock. | 1924 | conflict on any particular lock. |
1920 | 1925 | ||
1921 | 1926 | ||
1922 | LOCKS VS MEMORY ACCESSES | 1927 | ACQUIRES VS MEMORY ACCESSES |
1923 | ------------------------ | 1928 | --------------------------- |
1924 | 1929 | ||
1925 | Consider the following: the system has a pair of spinlocks (M) and (Q), and | 1930 | Consider the following: the system has a pair of spinlocks (M) and (Q), and |
1926 | three CPUs; then should the following sequence of events occur: | 1931 | three CPUs; then should the following sequence of events occur: |
@@ -1928,24 +1933,24 @@ three CPUs; then should the following sequence of events occur: | |||
1928 | CPU 1 CPU 2 | 1933 | CPU 1 CPU 2 |
1929 | =============================== =============================== | 1934 | =============================== =============================== |
1930 | ACCESS_ONCE(*A) = a; ACCESS_ONCE(*E) = e; | 1935 | ACCESS_ONCE(*A) = a; ACCESS_ONCE(*E) = e; |
1931 | LOCK M LOCK Q | 1936 | ACQUIRE M ACQUIRE Q |
1932 | ACCESS_ONCE(*B) = b; ACCESS_ONCE(*F) = f; | 1937 | ACCESS_ONCE(*B) = b; ACCESS_ONCE(*F) = f; |
1933 | ACCESS_ONCE(*C) = c; ACCESS_ONCE(*G) = g; | 1938 | ACCESS_ONCE(*C) = c; ACCESS_ONCE(*G) = g; |
1934 | UNLOCK M UNLOCK Q | 1939 | RELEASE M RELEASE Q |
1935 | ACCESS_ONCE(*D) = d; ACCESS_ONCE(*H) = h; | 1940 | ACCESS_ONCE(*D) = d; ACCESS_ONCE(*H) = h; |
1936 | 1941 | ||
1937 | Then there is no guarantee as to what order CPU 3 will see the accesses to *A | 1942 | Then there is no guarantee as to what order CPU 3 will see the accesses to *A |
1938 | through *H occur in, other than the constraints imposed by the separate locks | 1943 | through *H occur in, other than the constraints imposed by the separate locks |
1939 | on the separate CPUs. It might, for example, see: | 1944 | on the separate CPUs. It might, for example, see: |
1940 | 1945 | ||
1941 | *E, LOCK M, LOCK Q, *G, *C, *F, *A, *B, UNLOCK Q, *D, *H, UNLOCK M | 1946 | *E, ACQUIRE M, ACQUIRE Q, *G, *C, *F, *A, *B, RELEASE Q, *D, *H, RELEASE M |
1942 | 1947 | ||
1943 | But it won't see any of: | 1948 | But it won't see any of: |
1944 | 1949 | ||
1945 | *B, *C or *D preceding LOCK M | 1950 | *B, *C or *D preceding ACQUIRE M |
1946 | *A, *B or *C following UNLOCK M | 1951 | *A, *B or *C following RELEASE M |
1947 | *F, *G or *H preceding LOCK Q | 1952 | *F, *G or *H preceding ACQUIRE Q |
1948 | *E, *F or *G following UNLOCK Q | 1953 | *E, *F or *G following RELEASE Q |
1949 | 1954 | ||
1950 | 1955 | ||
1951 | However, if the following occurs: | 1956 | However, if the following occurs: |
@@ -1953,29 +1958,29 @@ However, if the following occurs: | |||
1953 | CPU 1 CPU 2 | 1958 | CPU 1 CPU 2 |
1954 | =============================== =============================== | 1959 | =============================== =============================== |
1955 | ACCESS_ONCE(*A) = a; | 1960 | ACCESS_ONCE(*A) = a; |
1956 | LOCK M [1] | 1961 | ACQUIRE M [1] |
1957 | ACCESS_ONCE(*B) = b; | 1962 | ACCESS_ONCE(*B) = b; |
1958 | ACCESS_ONCE(*C) = c; | 1963 | ACCESS_ONCE(*C) = c; |
1959 | UNLOCK M [1] | 1964 | RELEASE M [1] |
1960 | ACCESS_ONCE(*D) = d; ACCESS_ONCE(*E) = e; | 1965 | ACCESS_ONCE(*D) = d; ACCESS_ONCE(*E) = e; |
1961 | LOCK M [2] | 1966 | ACQUIRE M [2] |
1962 | smp_mb__after_unlock_lock(); | 1967 | smp_mb__after_unlock_lock(); |
1963 | ACCESS_ONCE(*F) = f; | 1968 | ACCESS_ONCE(*F) = f; |
1964 | ACCESS_ONCE(*G) = g; | 1969 | ACCESS_ONCE(*G) = g; |
1965 | UNLOCK M [2] | 1970 | RELEASE M [2] |
1966 | ACCESS_ONCE(*H) = h; | 1971 | ACCESS_ONCE(*H) = h; |
1967 | 1972 | ||
1968 | CPU 3 might see: | 1973 | CPU 3 might see: |
1969 | 1974 | ||
1970 | *E, LOCK M [1], *C, *B, *A, UNLOCK M [1], | 1975 | *E, ACQUIRE M [1], *C, *B, *A, RELEASE M [1], |
1971 | LOCK M [2], *H, *F, *G, UNLOCK M [2], *D | 1976 | ACQUIRE M [2], *H, *F, *G, RELEASE M [2], *D |
1972 | 1977 | ||
1973 | But assuming CPU 1 gets the lock first, CPU 3 won't see any of: | 1978 | But assuming CPU 1 gets the lock first, CPU 3 won't see any of: |
1974 | 1979 | ||
1975 | *B, *C, *D, *F, *G or *H preceding LOCK M [1] | 1980 | *B, *C, *D, *F, *G or *H preceding ACQUIRE M [1] |
1976 | *A, *B or *C following UNLOCK M [1] | 1981 | *A, *B or *C following RELEASE M [1] |
1977 | *F, *G or *H preceding LOCK M [2] | 1982 | *F, *G or *H preceding ACQUIRE M [2] |
1978 | *A, *B, *C, *E, *F or *G following UNLOCK M [2] | 1983 | *A, *B, *C, *E, *F or *G following RELEASE M [2] |
1979 | 1984 | ||
1980 | Note that the smp_mb__after_unlock_lock() is critically important | 1985 | Note that the smp_mb__after_unlock_lock() is critically important |
1981 | here: Without it CPU 3 might see some of the above orderings. | 1986 | here: Without it CPU 3 might see some of the above orderings. |
@@ -1983,8 +1988,8 @@ Without smp_mb__after_unlock_lock(), the accesses are not guaranteed | |||
1983 | to be seen in order unless CPU 3 holds lock M. | 1988 | to be seen in order unless CPU 3 holds lock M. |
1984 | 1989 | ||
1985 | 1990 | ||
1986 | LOCKS VS I/O ACCESSES | 1991 | ACQUIRES VS I/O ACCESSES |
1987 | --------------------- | 1992 | ------------------------ |
1988 | 1993 | ||
1989 | Under certain circumstances (especially involving NUMA), I/O accesses within | 1994 | Under certain circumstances (especially involving NUMA), I/O accesses within |
1990 | two spinlocked sections on two different CPUs may be seen as interleaved by the | 1995 | two spinlocked sections on two different CPUs may be seen as interleaved by the |
@@ -2202,13 +2207,13 @@ explicit lock operations, described later). These include: | |||
2202 | /* when succeeds (returns 1) */ | 2207 | /* when succeeds (returns 1) */ |
2203 | atomic_add_unless(); atomic_long_add_unless(); | 2208 | atomic_add_unless(); atomic_long_add_unless(); |
2204 | 2209 | ||
2205 | These are used for such things as implementing LOCK-class and UNLOCK-class | 2210 | These are used for such things as implementing ACQUIRE-class and RELEASE-class |
2206 | operations and adjusting reference counters towards object destruction, and as | 2211 | operations and adjusting reference counters towards object destruction, and as |
2207 | such the implicit memory barrier effects are necessary. | 2212 | such the implicit memory barrier effects are necessary. |
2208 | 2213 | ||
2209 | 2214 | ||
2210 | The following operations are potential problems as they do _not_ imply memory | 2215 | The following operations are potential problems as they do _not_ imply memory |
2211 | barriers, but might be used for implementing such things as UNLOCK-class | 2216 | barriers, but might be used for implementing such things as RELEASE-class |
2212 | operations: | 2217 | operations: |
2213 | 2218 | ||
2214 | atomic_set(); | 2219 | atomic_set(); |
@@ -2250,7 +2255,7 @@ The following operations are special locking primitives: | |||
2250 | clear_bit_unlock(); | 2255 | clear_bit_unlock(); |
2251 | __clear_bit_unlock(); | 2256 | __clear_bit_unlock(); |
2252 | 2257 | ||
2253 | These implement LOCK-class and UNLOCK-class operations. These should be used in | 2258 | These implement ACQUIRE-class and RELEASE-class operations. These should be used in |
2254 | preference to other operations when implementing locking primitives, because | 2259 | preference to other operations when implementing locking primitives, because |
2255 | their implementations can be optimised on many architectures. | 2260 | their implementations can be optimised on many architectures. |
2256 | 2261 | ||