blkcg: shoot down blkio_groups on elevator switch

Elevator switch may involve changes to blkcg policies.  Implement
shoot down of blkio_groups.

Combined with the previous bypass updates, the end goal is updating
blkcg core such that it can ensure that blkcg's being affected become
quiescent and don't have any per-blkg data hanging around before
commencing any policy updates.  Until queues are made aware of the
policies that applies to them, as an interim step, all per-policy blkg
data will be shot down.

* blk-throtl doesn't need this change as it can't be disabled for a
  live queue; however, update it anyway as the scheduled blkg
  unification requires this behavior change.  This means that
  blk-throtl configuration will be unnecessarily lost over elevator
  switch.  This oddity will be removed after blkcg learns to associate
  individual policies with request_queues.

* blk-throtl dosen't shoot down root_tg.  This is to ease transition.
  Unified blkg will always have persistent root group and not shooting
  down root_tg for now eases transition to that point by avoiding
  having to update td->root_tg and is safe as blk-throtl can never be
  disabled

-v2: Vivek pointed out that group list is not guaranteed to be empty
     on return from clear function if it raced cgroup removal and
     lost.  Fix it by waiting a bit and retrying.  This kludge will
     soon be removed once locking is updated such that blkg is never
     in limbo state between blkcg and request_queue locks.

     blk-throtl no longer shoots down root_tg to avoid breaking
     td->root_tg.

     Also, Nest queue_lock inside blkio_list_lock not the other way
     around to avoid introduce possible deadlock via blkcg lock.

-v3: blkcg_clear_queue() repositioned and renamed to
     blkg_destroy_all() to increase consistency with later changes.
     cfq_clear_queue() updated to check q->elevator before
     dereferencing it to avoid NULL dereference on not fully
     initialized queues (used by later change).

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Vivek Goyal <vgoyal@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

1 files changed, 25 insertions, 2 deletions

diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index 702c0e64e09f..3699ab40d494 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -989,12 +989,17 @@ throtl_destroy_tg(struct throtl_data *td, struct throtl_grp *tg)
         td->nr_undestroyed_grps--;
 }
 
-static void throtl_release_tgs(struct throtl_data *td)
+static bool throtl_release_tgs(struct throtl_data *td, bool release_root)
 {
         struct hlist_node *pos, *n;
         struct throtl_grp *tg;
+        bool empty = true;
 
         hlist_for_each_entry_safe(tg, pos, n, &td->tg_list, tg_node) {
+                /* skip root? */
+                if (!release_root && tg == td->root_tg)
+                        continue;
+
                 /*
                  * If cgroup removal path got to blk_group first and removed
                  * it from cgroup list, then it will take care of destroying
@@ -1002,7 +1007,10 @@ static void throtl_release_tgs(struct throtl_data *td)
                  */
                 if (!blkiocg_del_blkio_group(&tg->blkg))
                         throtl_destroy_tg(td, tg);
+                else
+                        empty = false;
         }
+        return empty;
 }
 
 /*
@@ -1029,6 +1037,20 @@ void throtl_unlink_blkio_group(void *key, struct blkio_group *blkg)
         spin_unlock_irqrestore(td->queue->queue_lock, flags);
 }
 
+static bool throtl_clear_queue(struct request_queue *q)
+{
+        lockdep_assert_held(q->queue_lock);
+
+        /*
+         * Clear tgs but leave the root one alone.  This is necessary
+         * because root_tg is expected to be persistent and safe because
+         * blk-throtl can never be disabled while @q is alive.  This is a
+         * kludge to prepare for unified blkg.  This whole function will be
+         * removed soon.
+         */
+        return throtl_release_tgs(q->td, false);
+}
+
 static void throtl_update_blkio_group_common(struct throtl_data *td,
                                 struct throtl_grp *tg)
 {
@@ -1097,6 +1119,7 @@ static void throtl_shutdown_wq(struct request_queue *q)
 static struct blkio_policy_type blkio_policy_throtl = {
         .ops = {
                 .blkio_unlink_group_fn = throtl_unlink_blkio_group,
+                .blkio_clear_queue_fn = throtl_clear_queue,
                 .blkio_update_group_read_bps_fn =
                                         throtl_update_blkio_group_read_bps,
                 .blkio_update_group_write_bps_fn =
@@ -1282,7 +1305,7 @@ void blk_throtl_exit(struct request_queue *q)
         throtl_shutdown_wq(q);
 
         spin_lock_irq(q->queue_lock);
-        throtl_release_tgs(td);
+        throtl_release_tgs(td, true);
 
         /* If there are other groups */
         if (td->nr_undestroyed_grps > 0)