jump label: Introduce static_branch() interface

Introduce:

static __always_inline bool static_branch(struct jump_label_key *key);

instead of the old JUMP_LABEL(key, label) macro.

In this way, jump labels become really easy to use:

Define:

        struct jump_label_key jump_key;

Can be used as:

        if (static_branch(&jump_key))
                do unlikely code

enable/disale via:

        jump_label_inc(&jump_key);
        jump_label_dec(&jump_key);

that's it!

For the jump labels disabled case, the static_branch() becomes an
atomic_read(), and jump_label_inc()/dec() are simply atomic_inc(),
atomic_dec() operations. We show testing results for this change below.

Thanks to H. Peter Anvin for suggesting the 'static_branch()' construct.

Since we now require a 'struct jump_label_key *key', we can store a pointer into
the jump table addresses. In this way, we can enable/disable jump labels, in
basically constant time. This change allows us to completely remove the previous
hashtable scheme. Thanks to Peter Zijlstra for this re-write.

Testing:

I ran a series of 'tbench 20' runs 5 times (with reboots) for 3
configurations, where tracepoints were disabled.

jump label configured in
avg: 815.6

jump label *not* configured in (using atomic reads)
avg: 800.1

jump label *not* configured in (regular reads)
avg: 803.4

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
LKML-Reference: <20110316212947.GA8792@redhat.com>
Signed-off-by: Jason Baron <jbaron@redhat.com>
Suggested-by: H. Peter Anvin <hpa@linux.intel.com>
Tested-by: David Daney <ddaney@caviumnetworks.com>
Acked-by: Ralf Baechle <ralf@linux-mips.org>
Acked-by: David S. Miller <davem@davemloft.net>
Acked-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>

1 files changed, 4 insertions, 10 deletions

diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h
index 32c45e5fe0ab..79522166d7f1 100644
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -170,6 +170,10 @@
         STRUCT_ALIGN();                                                 \
         *(__tracepoints)                                                \
         /* implement dynamic printk debug */                            \
+        . = ALIGN(8);                                                   \
+        VMLINUX_SYMBOL(__start___jump_table) = .;                       \
+        *(__jump_table)                                                 \
+        VMLINUX_SYMBOL(__stop___jump_table) = .;                        \
         . = ALIGN(8);                                                   \
         VMLINUX_SYMBOL(__start___verbose) = .;                          \
         *(__verbose)                                                    \
@@ -228,8 +232,6 @@
                                                                         \
         BUG_TABLE                                                       \
                                                                         \
-        JUMP_TABLE                                                      \
-                                                                        \
         /* PCI quirks */                                                \
         .pci_fixup        : AT(ADDR(.pci_fixup) - LOAD_OFFSET) {        \
                 VMLINUX_SYMBOL(__start_pci_fixups_early) = .;           \
@@ -589,14 +591,6 @@
 #define BUG_TABLE
 #endif
 
-#define JUMP_TABLE                                                      \
-        . = ALIGN(8);                                                   \
-        __jump_table : AT(ADDR(__jump_table) - LOAD_OFFSET) {           \
-                VMLINUX_SYMBOL(__start___jump_table) = .;               \
-                *(__jump_table)                                         \
-                VMLINUX_SYMBOL(__stop___jump_table) = .;                \
-        }
-
 #ifdef CONFIG_PM_TRACE
 #define TRACEDATA                                                       \
         . = ALIGN(4);                                                   \