block: fix diskstats access

There are two variants of stat functions - ones prefixed with double
underbars which don't care about preemption and ones without which
disable preemption before manipulating per-cpu counters.  It's unclear
whether the underbarred ones assume that preemtion is disabled on
entry as some callers don't do that.

This patch unifies diskstats access by implementing disk_stat_lock()
and disk_stat_unlock() which take care of both RCU (for partition
access) and preemption (for per-cpu counter access).  diskstats access
should always be enclosed between the two functions.  As such, there's
no need for the versions which disables preemption.  They're removed
and double underbars ones are renamed to drop the underbars.  As an
extra argument is added, there's no danger of using the old version
unconverted.

disk_stat_lock() uses get_cpu() and returns the cpu index and all
diskstat functions which access per-cpu counters now has @cpu
argument to help RT.

This change adds RCU or preemption operations at some places but also
collapses several preemption ops into one at others.  Overall, the
performance difference should be negligible as all involved ops are
very lightweight per-cpu ones.

Signed-off-by: Tejun Heo <tj@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Jens Axboe <jens.axboe@oracle.com>

1 files changed, 11 insertions, 9 deletions

diff --git a/block/genhd.c b/block/genhd.c
index b431d6543942..430626e440f0 100644
--- a/block/genhd.c
+++ b/block/genhd.c
@@ -633,10 +633,11 @@ static ssize_t disk_stat_show(struct device *dev,
                               struct device_attribute *attr, char *buf)
 {
         struct gendisk *disk = dev_to_disk(dev);
+        int cpu;
 
-        preempt_disable();
-        disk_round_stats(disk);
-        preempt_enable();
+        cpu = disk_stat_lock();
+        disk_round_stats(cpu, disk);
+        disk_stat_unlock();
         return sprintf(buf,
                 "%8lu %8lu %8llu %8u "
                 "%8lu %8lu %8llu %8u "
@@ -749,6 +750,7 @@ static int diskstats_show(struct seq_file *seqf, void *v)
         struct disk_part_iter piter;
         struct hd_struct *hd;
         char buf[BDEVNAME_SIZE];
+        int cpu;
 
         /*
         if (&gp->dev.kobj.entry == block_class.devices.next)
@@ -758,9 +760,9 @@ static int diskstats_show(struct seq_file *seqf, void *v)
                                 "\n\n");
         */
  
-        preempt_disable();
-        disk_round_stats(gp);
-        preempt_enable();
+        cpu = disk_stat_lock();
+        disk_round_stats(cpu, gp);
+        disk_stat_unlock();
         seq_printf(seqf, "%4d %4d %s %lu %lu %llu %u %lu %lu %llu %u %u %u %u\n",
                 MAJOR(disk_devt(gp)), MINOR(disk_devt(gp)),
                 disk_name(gp, 0, buf),
@@ -777,9 +779,9 @@ static int diskstats_show(struct seq_file *seqf, void *v)
         /* now show all non-0 size partitions of it */
         disk_part_iter_init(&piter, gp, 0);
         while ((hd = disk_part_iter_next(&piter))) {
-                preempt_disable();
-                part_round_stats(hd);
-                preempt_enable();
+                cpu = disk_stat_lock();
+                part_round_stats(cpu, hd);
+                disk_stat_unlock();
                 seq_printf(seqf, "%4d %4d %s %lu %lu %llu "
                            "%u %lu %lu %llu %u %u %u %u\n",
                            MAJOR(part_devt(hd)), MINOR(part_devt(hd)),