perf: Add AUX area to ring buffer for raw data streams

This patch introduces "AUX space" in the perf mmap buffer, intended for
exporting high bandwidth data streams to userspace, such as instruction
flow traces.

AUX space is a ring buffer, defined by aux_{offset,size} fields in the
user_page structure, and read/write pointers aux_{head,tail}, which abide
by the same rules as data_* counterparts of the main perf buffer.

In order to allocate/mmap AUX, userspace needs to set up aux_offset to
such an offset that will be greater than data_offset+data_size and
aux_size to be the desired buffer size. Both need to be page aligned.
Then, same aux_offset and aux_size should be passed to mmap() call and
if everything adds up, you should have an AUX buffer as a result.

Pages that are mapped into this buffer also come out of user's mlock
rlimit plus perf_event_mlock_kb allowance.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kaixu Xia <kaixu.xia@linaro.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Robert Richter <rric@kernel.org>
Cc: Stephane Eranian <eranian@google.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: acme@infradead.org
Cc: adrian.hunter@intel.com
Cc: kan.liang@intel.com
Cc: markus.t.metzger@intel.com
Cc: mathieu.poirier@linaro.org
Link: http://lkml.kernel.org/r/1421237903-181015-3-git-send-email-alexander.shishkin@linux.intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>

2 files changed, 33 insertions, 0 deletions

diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 401554074de9..5a94f6d6fa91 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -284,6 +284,18 @@ struct pmu {
          * Return the count value for a counter.
          */
         u64 (*count)                    (struct perf_event *event); /*optional*/
+
+        /*
+         * Set up pmu-private data structures for an AUX area
+         */
+        void *(*setup_aux)              (int cpu, void **pages,
+                                         int nr_pages, bool overwrite);
+                                        /* optional */
+
+        /*
+         * Free pmu-private AUX data structures
+         */
+        void (*free_aux)                (void *aux); /* optional */
 };
 
 /**
@@ -862,6 +874,11 @@ static inline bool needs_branch_stack(struct perf_event *event)
         return event->attr.branch_sample_type != 0;
 }
 
+static inline bool has_aux(struct perf_event *event)
+{
+        return event->pmu->setup_aux;
+}
+
 extern int perf_output_begin(struct perf_output_handle *handle,
                              struct perf_event *event, unsigned int size);
 extern void perf_output_end(struct perf_output_handle *handle);
diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
index 86c44ae66d43..6c5013a71714 100644
--- a/include/uapi/linux/perf_event.h
+++ b/include/uapi/linux/perf_event.h
@@ -530,6 +530,22 @@ struct perf_event_mmap_page {
         __u64   data_tail;              /* user-space written tail */
         __u64   data_offset;            /* where the buffer starts */
         __u64   data_size;              /* data buffer size */
+
+        /*
+         * AUX area is defined by aux_{offset,size} fields that should be set
+         * by the userspace, so that
+         *
+         *   aux_offset >= data_offset + data_size
+         *
+         * prior to mmap()ing it. Size of the mmap()ed area should be aux_size.
+         *
+         * Ring buffer pointers aux_{head,tail} have the same semantics as
+         * data_{head,tail} and same ordering rules apply.
+         */
+        __u64   aux_head;
+        __u64   aux_tail;
+        __u64   aux_offset;
+        __u64   aux_size;
 };
 
 #define PERF_RECORD_MISC_CPUMODE_MASK           (7 << 0)