Btrfs: fix the deadlock between the transaction start/attach and commit

Now btrfs_commit_transaction() does this

ret = btrfs_run_ordered_operations(root, 0)

which async flushes all inodes on the ordered operations list, it introduced
a deadlock that transaction-start task, transaction-commit task and the flush
workers waited for each other.
(See the following URL to get the detail
 http://marc.info/?l=linux-btrfs&m=136070705732646&w=2)

As we know, if ->in_commit is set, it means someone is committing the
current transaction, we should not try to join it if we are not JOIN
or JOIN_NOLOCK, wait is the best choice for it. In this way, we can avoid
the above problem. In this way, there is another benefit: there is no new
transaction handle to block the transaction which is on the way of commit,
once we set ->in_commit.

Signed-off-by: Miao Xie <miaox@cn.fujitsu.com>
Signed-off-by: Josef Bacik <jbacik@fusionio.com>

1 files changed, 16 insertions, 1 deletions

diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 425d5b57d377..5767ea1c0150 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -50,6 +50,14 @@ static noinline void switch_commit_root(struct btrfs_root *root)
         root->commit_root = btrfs_root_node(root);
 }
 
+static inline int can_join_transaction(struct btrfs_transaction *trans,
+                                       int type)
+{
+        return !(trans->in_commit &&
+                 type != TRANS_JOIN &&
+                 type != TRANS_JOIN_NOLOCK);
+}
+
 /*
  * either allocate a new transaction or hop into the existing one
  */
@@ -85,6 +93,10 @@ loop:
                         spin_unlock(&fs_info->trans_lock);
                         return cur_trans->aborted;
                 }
+                if (!can_join_transaction(cur_trans, type)) {
+                        spin_unlock(&fs_info->trans_lock);
+                        return -EBUSY;
+                }
                 atomic_inc(&cur_trans->use_count);
                 atomic_inc(&cur_trans->num_writers);
                 cur_trans->num_joined++;
@@ -360,8 +372,11 @@ again:
 
         do {
                 ret = join_transaction(root, type);
-                if (ret == -EBUSY)
+                if (ret == -EBUSY) {
                         wait_current_trans(root);
+                        if (unlikely(type == TRANS_ATTACH))
+                                ret = -ENOENT;
+                }
         } while (ret == -EBUSY);
 
         if (ret < 0) {