perf, trace: Optimize tracepoints by using per-tracepoint-per-cpu hlist to track events

Avoid the swevent hash-table by using per-tracepoint
hlists.

Also, avoid conditionals on the fast path by ordering
with probe unregister so that we should never get on
the callback path without the data being there.

Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
LKML-Reference: <20100521090710.473188012@chello.nl>
Signed-off-by: Ingo Molnar <mingo@elte.hu>

1 files changed, 3 insertions, 1 deletions

diff --git a/include/trace/ftrace.h b/include/trace/ftrace.h
index f282885057dd..4eb2148f1321 100644
--- a/include/trace/ftrace.h
+++ b/include/trace/ftrace.h
@@ -768,6 +768,7 @@ perf_trace_templ_##call(struct ftrace_event_call *event_call,		\
         struct ftrace_data_offsets_##call __maybe_unused __data_offsets;\
         struct ftrace_raw_##call *entry;                                \
         u64 __addr = 0, __count = 1;                                    \
+        struct hlist_head *head;                                        \
         int __entry_size;                                               \
         int __data_size;                                                \
         int rctx;                                                       \
@@ -790,8 +791,9 @@ perf_trace_templ_##call(struct ftrace_event_call *event_call,		\
                                                                         \
         { assign; }                                                     \
                                                                         \
+        head = per_cpu_ptr(event_call->perf_events, smp_processor_id());\
         perf_trace_buf_submit(entry, __entry_size, rctx, __addr,        \
-                               __count, __regs, event_call->perf_data); \
+                __count, __regs, head);                                 \
 }
 
 #undef DEFINE_EVENT