block: hold queue if flush is running for non-queueable flush drive

In some drives, flush requests are non-queueable. When flush request is
running, normal read/write requests can't run. If block layer dispatches
such request, driver can't handle it and requeue it.  Tejun suggested we
can hold the queue when flush is running. This can avoid unnecessary
requeue.  Also this can improve performance. For example, we have
request flush1, write1, flush 2. flush1 is dispatched, then queue is
hold, write1 isn't inserted to queue. After flush1 is finished, flush2
will be dispatched. Since disk cache is already clean, flush2 will be
finished very soon, so looks like flush2 is folded to flush1.

In my test, the queue holding completely solves a regression introduced by
commit 53d63e6b0dfb95882ec0219ba6bbd50cde423794:

    block: make the flush insertion use the tail of the dispatch list

    It's not a preempt type request, in fact we have to insert it
    behind requests that do specify INSERT_FRONT.

which causes about 20% regression running a sysbench fileio
workload.

Stable: 2.6.39 only

Cc: stable@kernel.org
Signed-off-by: Shaohua Li <shaohua.li@intel.com>
Acked-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Jens Axboe <jaxboe@fusionio.com>

1 files changed, 20 insertions, 1 deletions

diff --git a/block/blk.h b/block/blk.h
index c9df8fc3c999..83e4bff36201 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -62,7 +62,26 @@ static inline struct request *__elv_next_request(struct request_queue *q)
                         rq = list_entry_rq(q->queue_head.next);
                         return rq;
                 }
-
+                /*
+                 * Flush request is running and flush request isn't queueable
+                 * in the drive, we can hold the queue till flush request is
+                 * finished. Even we don't do this, driver can't dispatch next
+                 * requests and will requeue them. And this can improve
+                 * throughput too. For example, we have request flush1, write1,
+                 * flush 2. flush1 is dispatched, then queue is hold, write1
+                 * isn't inserted to queue. After flush1 is finished, flush2
+                 * will be dispatched. Since disk cache is already clean,
+                 * flush2 will be finished very soon, so looks like flush2 is
+                 * folded to flush1.
+                 * Since the queue is hold, a flag is set to indicate the queue
+                 * should be restarted later. Please see flush_end_io() for
+                 * details.
+                 */
+                if (q->flush_pending_idx != q->flush_running_idx &&
+                                !queue_flush_queueable(q)) {
+                        q->flush_queue_delayed = 1;
+                        return NULL;
+                }
                 if (!q->elevator->ops->elevator_dispatch_fn(q, 0))
                         return NULL;
         }