percpu_ref: make INIT_ATOMIC and switch_to_atomic() sticky

Currently, a percpu_ref which is initialized with
PERPCU_REF_INIT_ATOMIC or switched to atomic mode via
switch_to_atomic() automatically reverts to percpu mode on the first
percpu_ref_reinit().  This makes the atomic mode difficult to use for
cases where a percpu_ref is used as a persistent on/off switch which
may be cycled multiple times.

This patch makes such atomic state sticky so that it survives through
kill/reinit cycles.  After this patch, atomic state is cleared only by
an explicit percpu_ref_switch_to_percpu() call.

Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Kent Overstreet <kmo@daterainc.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>

1 files changed, 4 insertions, 1 deletions

diff --git a/include/linux/percpu-refcount.h b/include/linux/percpu-refcount.h
index b0293f268cd2..00c01fc6d88c 100644
--- a/include/linux/percpu-refcount.h
+++ b/include/linux/percpu-refcount.h
@@ -67,7 +67,9 @@ enum {
 enum {
         /*
          * Start w/ ref == 1 in atomic mode.  Can be switched to percpu
-         * operation using percpu_ref_switch_to_percpu().
+         * operation using percpu_ref_switch_to_percpu().  If initialized
+         * with this flag, the ref will stay in atomic mode until
+         * percpu_ref_switch_to_percpu() is invoked on it.
          */
         PERCPU_REF_INIT_ATOMIC  = 1 << 0,
 
@@ -87,6 +89,7 @@ struct percpu_ref {
         unsigned long           percpu_count_ptr;
         percpu_ref_func_t       *release;
         percpu_ref_func_t       *confirm_switch;
+        bool                    force_atomic:1;
         struct rcu_head         rcu;
 };