[TG3]: Fix tx race condition

Fix a subtle race condition between tg3_start_xmit() and tg3_tx()
discovered by Herbert Xu <herbert@gondor.apana.org.au>:

CPU0					CPU1
tg3_start_xmit()
	if (tx_ring_full) {
		tx_lock
					tg3_tx()
						if (!netif_queue_stopped)
		netif_stop_queue()
		if (!tx_ring_full)
						update_tx_ring 
			netif_wake_queue()
		tx_unlock
	}

Even though tx_ring is updated before the if statement in tg3_tx() in
program order, it can be re-ordered by the CPU as shown above.  This
scenario can cause the tx queue to be stopped forever if tg3_tx() has
just freed up the entire tx_ring.  The possibility of this happening
should be very rare though.

The following changes are made:

1. Add memory barrier to fix the above race condition.

2. Eliminate the private tx_lock altogether and rely solely on
netif_tx_lock.  This eliminates one spinlock in tg3_start_xmit()
when the ring is full.

3. Because of 2, use netif_tx_lock in tg3_tx() before calling
netif_wake_queue().

4. Change TX_BUFFS_AVAIL to an inline function with a memory barrier.
Herbert and David suggested using the memory barrier instead of
volatile.

5. Check for the full wake queue condition before getting
netif_tx_lock in tg3_tx().  This reduces the number of unnecessary
spinlocks when the tx ring is full in a steady-state condition.

6. Update version to 3.65.

Signed-off-by: Michael Chan <mchan@broadcom.com>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: David S. Miller <davem@davemloft.net>

1 files changed, 3 insertions, 5 deletions

diff --git a/drivers/net/tg3.h b/drivers/net/tg3.h
index ba2c98711c88..3ecf356cfb08 100644
--- a/drivers/net/tg3.h
+++ b/drivers/net/tg3.h
@@ -2079,9 +2079,9 @@ struct tg3 {
          * lock: Held during reset, PHY access, timer, and when
          *       updating tg3_flags and tg3_flags2.
          *
-         * tx_lock: Held during tg3_start_xmit and tg3_tx only
-         *          when calling netif_[start|stop]_queue.
-         *          tg3_start_xmit is protected by netif_tx_lock.
+         * netif_tx_lock: Held during tg3_start_xmit. tg3_tx holds
+         *                netif_tx_lock when it needs to call
+         *                netif_wake_queue.
          *
          * Both of these locks are to be held with BH safety.
          *
@@ -2118,8 +2118,6 @@ struct tg3 {
         u32                             tx_cons;
         u32                             tx_pending;
 
-        spinlock_t                      tx_lock;
-
         struct tg3_tx_buffer_desc       *tx_ring;
         struct tx_ring_info             *tx_buffers;
         dma_addr_t                      tx_desc_mapping;