Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

Pull perf updates from Ingo Molnar:
 "Features:

   - Add "uretprobes" - an optimization to uprobes, like kretprobes are
     an optimization to kprobes.  "perf probe -x file sym%return" now
     works like kretprobes.  By Oleg Nesterov.

   - Introduce per core aggregation in 'perf stat', from Stephane
     Eranian.

   - Add memory profiling via PEBS, from Stephane Eranian.

   - Event group view for 'annotate' in --stdio, --tui and --gtk, from
     Namhyung Kim.

   - Add support for AMD NB and L2I "uncore" counters, by Jacob Shin.

   - Add Ivy Bridge-EP uncore support, by Zheng Yan

   - IBM zEnterprise EC12 oprofile support patchlet from Robert Richter.

   - Add perf test entries for checking breakpoint overflow signal
     handler issues, from Jiri Olsa.

   - Add perf test entry for for checking number of EXIT events, from
     Namhyung Kim.

   - Add perf test entries for checking --cpu in record and stat, from
     Jiri Olsa.

   - Introduce perf stat --repeat forever, from Frederik Deweerdt.

   - Add --no-demangle to report/top, from Namhyung Kim.

   - PowerPC fixes plus a couple of cleanups/optimizations in uprobes
     and trace_uprobes, by Oleg Nesterov.

  Various fixes and refactorings:

   - Fix dependency of the python binding wrt libtraceevent, from
     Naohiro Aota.

   - Simplify some perf_evlist methods and to allow 'stat' to share code
     with 'record' and 'trace', by Arnaldo Carvalho de Melo.

   - Remove dead code in related to libtraceevent integration, from
     Namhyung Kim.

   - Revert "perf sched: Handle PERF_RECORD_EXIT events" to get 'perf
     sched lat' back working, by Arnaldo Carvalho de Melo

   - We don't use Newt anymore, just plain libslang, by Arnaldo Carvalho
     de Melo.

   - Kill a bunch of die() calls, from Namhyung Kim.

   - Fix build on non-glibc systems due to libio.h absence, from Cody P
     Schafer.

   - Remove some perf_session and tracing dead code, from David Ahern.

   - Honor parallel jobs, fix from Borislav Petkov

   - Introduce tools/lib/lk library, initially just removing duplication
     among tools/perf and tools/vm.  from Borislav Petkov

  ... and many more I missed to list, see the shortlog and git log for
  more details."

* 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (136 commits)
  perf/x86/intel/P4: Robistify P4 PMU types
  perf/x86/amd: Fix AMD NB and L2I "uncore" support
  perf/x86/amd: Remove old-style NB counter support from perf_event_amd.c
  perf/x86: Check all MSRs before passing hw check
  perf/x86/amd: Add support for AMD NB and L2I "uncore" counters
  perf/x86/intel: Add Ivy Bridge-EP uncore support
  perf/x86/intel: Fix SNB-EP CBO and PCU uncore PMU filter management
  perf/x86: Avoid kfree() in CPU_{STARTING,DYING}
  uprobes/perf: Avoid perf_trace_buf_prepare/submit if ->perf_events is empty
  uprobes/tracing: Don't pass addr=ip to perf_trace_buf_submit()
  uprobes/tracing: Change create_trace_uprobe() to support uretprobes
  uprobes/tracing: Make seq_printf() code uretprobe-friendly
  uprobes/tracing: Make register_uprobe_event() paths uretprobe-friendly
  uprobes/tracing: Make uprobe_{trace,perf}_print() uretprobe-friendly
  uprobes/tracing: Introduce is_ret_probe() and uretprobe_dispatcher()
  uprobes/tracing: Introduce uprobe_{trace,perf}_print() helpers
  uprobes/tracing: Generalize struct uprobe_trace_entry_head
  uprobes/tracing: Kill the pointless local_save_flags/preempt_count calls
  uprobes/tracing: Kill the pointless seq_print_ip_sym() call
  uprobes/tracing: Kill the pointless task_pt_regs() calls
  ...

130 files changed, 5670 insertions, 1708 deletions

diff --git a/Documentation/trace/uprobetracer.txt b/Documentation/trace/uprobetracer.txt
index 24ce6823a09e..d9c3e682312c 100644
--- a/Documentation/trace/uprobetracer.txt
+++ b/Documentation/trace/uprobetracer.txt
@@ -1,6 +1,8 @@
-                Uprobe-tracer: Uprobe-based Event Tracing
-                =========================================
-                 Documentation written by Srikar Dronamraju
+            Uprobe-tracer: Uprobe-based Event Tracing
+            =========================================
+
+           Documentation written by Srikar Dronamraju
+
 
 Overview
 --------
@@ -13,78 +15,94 @@ current_tracer. Instead of that, add probe points via
 /sys/kernel/debug/tracing/events/uprobes/<EVENT>/enabled.
 
 However unlike kprobe-event tracer, the uprobe event interface expects the
-user to calculate the offset of the probepoint in the object
+user to calculate the offset of the probepoint in the object.
 
 Synopsis of uprobe_tracer
 -------------------------
-  p[:[GRP/]EVENT] PATH:SYMBOL[+offs] [FETCHARGS]        : Set a probe
+  p[:[GRP/]EVENT] PATH:SYMBOL[+offs] [FETCHARGS] : Set a uprobe
+  r[:[GRP/]EVENT] PATH:SYMBOL[+offs] [FETCHARGS] : Set a return uprobe (uretprobe)
+  -:[GRP/]EVENT                                  : Clear uprobe or uretprobe event
 
- GRP            : Group name. If omitted, use "uprobes" for it.
- EVENT          : Event name. If omitted, the event name is generated
-                  based on SYMBOL+offs.
- PATH           : path to an executable or a library.
- SYMBOL[+offs]  : Symbol+offset where the probe is inserted.
+  GRP           : Group name. If omitted, "uprobes" is the default value.
+  EVENT         : Event name. If omitted, the event name is generated based
+                  on SYMBOL+offs.
+  PATH          : Path to an executable or a library.
+  SYMBOL[+offs] : Symbol+offset where the probe is inserted.
 
- FETCHARGS      : Arguments. Each probe can have up to 128 args.
-  %REG          : Fetch register REG
+  FETCHARGS     : Arguments. Each probe can have up to 128 args.
+   %REG         : Fetch register REG
 
 Event Profiling
 ---------------
- You can check the total number of probe hits and probe miss-hits via
+You can check the total number of probe hits and probe miss-hits via
 /sys/kernel/debug/tracing/uprobe_profile.
- The first column is event name, the second is the number of probe hits,
+The first column is event name, the second is the number of probe hits,
 the third is the number of probe miss-hits.
 
 Usage examples
 --------------
-To add a probe as a new event, write a new definition to uprobe_events
-as below.
+ * Add a probe as a new uprobe event, write a new definition to uprobe_events
+as below: (sets a uprobe at an offset of 0x4245c0 in the executable /bin/bash)
+
+    echo 'p: /bin/bash:0x4245c0' > /sys/kernel/debug/tracing/uprobe_events
+
+ * Add a probe as a new uretprobe event:
+
+    echo 'r: /bin/bash:0x4245c0' > /sys/kernel/debug/tracing/uprobe_events
+
+ * Unset registered event:
 
-  echo 'p: /bin/bash:0x4245c0' > /sys/kernel/debug/tracing/uprobe_events
+    echo '-:bash_0x4245c0' >> /sys/kernel/debug/tracing/uprobe_events
 
- This sets a uprobe at an offset of 0x4245c0 in the executable /bin/bash
+ * Print out the events that are registered:
 
-  echo > /sys/kernel/debug/tracing/uprobe_events
+    cat /sys/kernel/debug/tracing/uprobe_events
 
- This clears all probe points.
+ * Clear all events:
 
-The following example shows how to dump the instruction pointer and %ax
-a register at the probed text address.  Here we are trying to probe
-function zfree in /bin/zsh
+    echo > /sys/kernel/debug/tracing/uprobe_events
+
+Following example shows how to dump the instruction pointer and %ax register
+at the probed text address. Probe zfree function in /bin/zsh:
 
     # cd /sys/kernel/debug/tracing/
-    # cat /proc/`pgrep  zsh`/maps | grep /bin/zsh | grep r-xp
+    # cat /proc/`pgrep zsh`/maps | grep /bin/zsh | grep r-xp
     00400000-0048a000 r-xp 00000000 08:03 130904 /bin/zsh
     # objdump -T /bin/zsh | grep -w zfree
     0000000000446420 g    DF .text  0000000000000012  Base        zfree
 
-0x46420 is the offset of zfree in object /bin/zsh that is loaded at
-0x00400000. Hence the command to probe would be :
+  0x46420 is the offset of zfree in object /bin/zsh that is loaded at
+  0x00400000. Hence the command to uprobe would be:
+
+    # echo 'p:zfree_entry /bin/zsh:0x46420 %ip %ax' > uprobe_events
+
+  And the same for the uretprobe would be:
 
-    # echo 'p /bin/zsh:0x46420 %ip %ax' > uprobe_events
+    # echo 'r:zfree_exit /bin/zsh:0x46420 %ip %ax' >> uprobe_events
 
-Please note: User has to explicitly calculate the offset of the probepoint
+Please note: User has to explicitly calculate the offset of the probe-point
 in the object. We can see the events that are registered by looking at the
 uprobe_events file.
 
     # cat uprobe_events
-    p:uprobes/p_zsh_0x46420 /bin/zsh:0x00046420 arg1=%ip arg2=%ax
+    p:uprobes/zfree_entry /bin/zsh:0x00046420 arg1=%ip arg2=%ax
+    r:uprobes/zfree_exit /bin/zsh:0x00046420 arg1=%ip arg2=%ax
 
-The format of events can be seen by viewing the file events/uprobes/p_zsh_0x46420/format
+Format of events can be seen by viewing the file events/uprobes/zfree_entry/format
 
-    # cat events/uprobes/p_zsh_0x46420/format
-    name: p_zsh_0x46420
+    # cat events/uprobes/zfree_entry/format
+    name: zfree_entry
     ID: 922
     format:
-        field:unsigned short common_type;       offset:0;       size:2; signed:0;
-        field:unsigned char common_flags;       offset:2;       size:1; signed:0;
-        field:unsigned char common_preempt_count;       offset:3;       size:1; signed:0;
-        field:int common_pid;   offset:4;       size:4; signed:1;
-        field:int common_padding;       offset:8;       size:4; signed:1;
+         field:unsigned short common_type;         offset:0;  size:2; signed:0;
+         field:unsigned char common_flags;         offset:2;  size:1; signed:0;
+         field:unsigned char common_preempt_count; offset:3;  size:1; signed:0;
+         field:int common_pid;                     offset:4;  size:4; signed:1;
+         field:int common_padding;                 offset:8;  size:4; signed:1;
 
-        field:unsigned long __probe_ip; offset:12;      size:4; signed:0;
-        field:u32 arg1; offset:16;      size:4; signed:0;
-        field:u32 arg2; offset:20;      size:4; signed:0;
+         field:unsigned long __probe_ip;           offset:12; size:4; signed:0;
+         field:u32 arg1;                           offset:16; size:4; signed:0;
+         field:u32 arg2;                           offset:20; size:4; signed:0;
 
     print fmt: "(%lx) arg1=%lx arg2=%lx", REC->__probe_ip, REC->arg1, REC->arg2
 
@@ -94,6 +112,7 @@ events, you need to enable it by:
     # echo 1 > events/uprobes/enable
 
 Lets disable the event after sleeping for some time.
+
     # sleep 20
     # echo 0 > events/uprobes/enable
 
@@ -104,10 +123,11 @@ And you can see the traced information via /sys/kernel/debug/tracing/trace.
     #
     #           TASK-PID    CPU#    TIMESTAMP  FUNCTION
     #              | |       |          |         |
-                 zsh-24842 [006] 258544.995456: p_zsh_0x46420: (0x446420) arg1=446421 arg2=79
-                 zsh-24842 [007] 258545.000270: p_zsh_0x46420: (0x446420) arg1=446421 arg2=79
-                 zsh-24842 [002] 258545.043929: p_zsh_0x46420: (0x446420) arg1=446421 arg2=79
-                 zsh-24842 [004] 258547.046129: p_zsh_0x46420: (0x446420) arg1=446421 arg2=79
-
-Each line shows us probes were triggered for a pid 24842 with ip being
-0x446421 and contents of ax register being 79.
+                 zsh-24842 [006] 258544.995456: zfree_entry: (0x446420) arg1=446420 arg2=79
+                 zsh-24842 [007] 258545.000270: zfree_exit:  (0x446540 <- 0x446420) arg1=446540 arg2=0
+                 zsh-24842 [002] 258545.043929: zfree_entry: (0x446420) arg1=446420 arg2=79
+                 zsh-24842 [004] 258547.046129: zfree_exit:  (0x446540 <- 0x446420) arg1=446540 arg2=0
+
+Output shows us uprobe was triggered for a pid 24842 with ip being 0x446420
+and contents of ax register being 79. And uretprobe was triggered with ip at
+0x446540 with counterpart function entry at 0x446420.
diff --git a/Makefile b/Makefile
index 8fe69916e726..d3528f6aa938 100644
--- a/Makefile
+++ b/Makefile
@@ -1332,11 +1332,11 @@ kernelversion:
 # Clear a bunch of variables before executing the submake
 tools/: FORCE
         $(Q)mkdir -p $(objtree)/tools
-        $(Q)$(MAKE) LDFLAGS= MAKEFLAGS= O=$(objtree) subdir=tools -C $(src)/tools/
+        $(Q)$(MAKE) LDFLAGS= MAKEFLAGS="$(filter --j% -j,$(MAKEFLAGS))" O=$(objtree) subdir=tools -C $(src)/tools/
 
 tools/%: FORCE
         $(Q)mkdir -p $(objtree)/tools
-        $(Q)$(MAKE) LDFLAGS= MAKEFLAGS= O=$(objtree) subdir=tools -C $(src)/tools/ $*
+        $(Q)$(MAKE) LDFLAGS= MAKEFLAGS="$(filter --j% -j,$(MAKEFLAGS))" O=$(objtree) subdir=tools -C $(src)/tools/ $*
 
 # Single targets
 # ---------------------------------------------------------------------------
diff --git a/arch/powerpc/include/asm/uprobes.h b/arch/powerpc/include/asm/uprobes.h
index b532060d0916..23016020915e 100644
--- a/arch/powerpc/include/asm/uprobes.h
+++ b/arch/powerpc/include/asm/uprobes.h
@@ -51,4 +51,5 @@ extern int  arch_uprobe_post_xol(struct arch_uprobe *aup, struct pt_regs *regs);
 extern bool arch_uprobe_xol_was_trapped(struct task_struct *tsk);
 extern int  arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data);
 extern void arch_uprobe_abort_xol(struct arch_uprobe *aup, struct pt_regs *regs);
+extern unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs);
 #endif  /* _ASM_UPROBES_H */
diff --git a/arch/powerpc/kernel/uprobes.c b/arch/powerpc/kernel/uprobes.c
index bc77834dbf43..59f419b935f2 100644
--- a/arch/powerpc/kernel/uprobes.c
+++ b/arch/powerpc/kernel/uprobes.c
@@ -31,6 +31,16 @@
 #define UPROBE_TRAP_NR  UINT_MAX
 
 /**
+ * is_trap_insn - check if the instruction is a trap variant
+ * @insn: instruction to be checked.
+ * Returns true if @insn is a trap variant.
+ */
+bool is_trap_insn(uprobe_opcode_t *insn)
+{
+        return (is_trap(*insn));
+}
+
+/**
  * arch_uprobe_analyze_insn
  * @mm: the probed address space.
  * @arch_uprobe: the probepoint information.
@@ -43,12 +53,6 @@ int arch_uprobe_analyze_insn(struct arch_uprobe *auprobe,
         if (addr & 0x03)
                 return -EINVAL;
 
-        /*
-         * We currently don't support a uprobe on an already
-         * existing breakpoint instruction underneath
-         */
-        if (is_trap(auprobe->ainsn))
-                return -ENOTSUPP;
         return 0;
 }
 
@@ -188,3 +192,16 @@ bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
 
         return false;
 }
+
+unsigned long
+arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs)
+{
+        unsigned long orig_ret_vaddr;
+
+        orig_ret_vaddr = regs->link;
+
+        /* Replace the return addr with trampoline addr */
+        regs->link = trampoline_vaddr;
+
+        return orig_ret_vaddr;
+}
diff --git a/arch/s390/oprofile/init.c b/arch/s390/oprofile/init.c
index 584b93674ea4..ffeb17ce7f31 100644
--- a/arch/s390/oprofile/init.c
+++ b/arch/s390/oprofile/init.c
@@ -440,6 +440,7 @@ static int oprofile_hwsampler_init(struct oprofile_operations *ops)
                 switch (id.machine) {
                 case 0x2097: case 0x2098: ops->cpu_type = "s390/z10"; break;
                 case 0x2817: case 0x2818: ops->cpu_type = "s390/z196"; break;
+                case 0x2827:              ops->cpu_type = "s390/zEC12"; break;
                 default: return -ENODEV;
                 }
         }
diff --git a/arch/x86/include/asm/cpufeature.h b/arch/x86/include/asm/cpufeature.h
index 93fe929d1cee..ac10df72925b 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -168,6 +168,7 @@
 #define X86_FEATURE_TOPOEXT     (6*32+22) /* topology extensions CPUID leafs */
 #define X86_FEATURE_PERFCTR_CORE (6*32+23) /* core performance counter extensions */
 #define X86_FEATURE_PERFCTR_NB  (6*32+24) /* NB performance counter extensions */
+#define X86_FEATURE_PERFCTR_L2  (6*32+28) /* L2 performance counter extensions */
 
 /*
  * Auxiliary flags: Linux defined - For features scattered in various
@@ -311,6 +312,7 @@ extern const char * const x86_power_flags[32];
 #define cpu_has_pclmulqdq       boot_cpu_has(X86_FEATURE_PCLMULQDQ)
 #define cpu_has_perfctr_core    boot_cpu_has(X86_FEATURE_PERFCTR_CORE)
 #define cpu_has_perfctr_nb      boot_cpu_has(X86_FEATURE_PERFCTR_NB)
+#define cpu_has_perfctr_l2      boot_cpu_has(X86_FEATURE_PERFCTR_L2)
 #define cpu_has_cx8             boot_cpu_has(X86_FEATURE_CX8)
 #define cpu_has_cx16            boot_cpu_has(X86_FEATURE_CX16)
 #define cpu_has_eager_fpu       boot_cpu_has(X86_FEATURE_EAGER_FPU)
diff --git a/arch/x86/include/asm/perf_event_p4.h b/arch/x86/include/asm/perf_event_p4.h
index 4f7e67e2345e..85e13ccf15c4 100644
--- a/arch/x86/include/asm/perf_event_p4.h
+++ b/arch/x86/include/asm/perf_event_p4.h
@@ -24,45 +24,45 @@
 #define ARCH_P4_CNTRVAL_MASK    ((1ULL << ARCH_P4_CNTRVAL_BITS) - 1)
 #define ARCH_P4_UNFLAGGED_BIT   ((1ULL) << (ARCH_P4_CNTRVAL_BITS - 1))
 
-#define P4_ESCR_EVENT_MASK      0x7e000000U
+#define P4_ESCR_EVENT_MASK      0x7e000000ULL
 #define P4_ESCR_EVENT_SHIFT     25
-#define P4_ESCR_EVENTMASK_MASK  0x01fffe00U
+#define P4_ESCR_EVENTMASK_MASK  0x01fffe00ULL
 #define P4_ESCR_EVENTMASK_SHIFT 9
-#define P4_ESCR_TAG_MASK        0x000001e0U
+#define P4_ESCR_TAG_MASK        0x000001e0ULL
 #define P4_ESCR_TAG_SHIFT       5
-#define P4_ESCR_TAG_ENABLE      0x00000010U
-#define P4_ESCR_T0_OS           0x00000008U
-#define P4_ESCR_T0_USR          0x00000004U
-#define P4_ESCR_T1_OS           0x00000002U
-#define P4_ESCR_T1_USR          0x00000001U
+#define P4_ESCR_TAG_ENABLE      0x00000010ULL
+#define P4_ESCR_T0_OS           0x00000008ULL
+#define P4_ESCR_T0_USR          0x00000004ULL
+#define P4_ESCR_T1_OS           0x00000002ULL
+#define P4_ESCR_T1_USR          0x00000001ULL
 
 #define P4_ESCR_EVENT(v)        ((v) << P4_ESCR_EVENT_SHIFT)
 #define P4_ESCR_EMASK(v)        ((v) << P4_ESCR_EVENTMASK_SHIFT)
 #define P4_ESCR_TAG(v)          ((v) << P4_ESCR_TAG_SHIFT)
 
-#define P4_CCCR_OVF                     0x80000000U
-#define P4_CCCR_CASCADE                 0x40000000U
-#define P4_CCCR_OVF_PMI_T0              0x04000000U
-#define P4_CCCR_OVF_PMI_T1              0x08000000U
-#define P4_CCCR_FORCE_OVF               0x02000000U
-#define P4_CCCR_EDGE                    0x01000000U
-#define P4_CCCR_THRESHOLD_MASK          0x00f00000U
+#define P4_CCCR_OVF                     0x80000000ULL
+#define P4_CCCR_CASCADE                 0x40000000ULL
+#define P4_CCCR_OVF_PMI_T0              0x04000000ULL
+#define P4_CCCR_OVF_PMI_T1              0x08000000ULL
+#define P4_CCCR_FORCE_OVF               0x02000000ULL
+#define P4_CCCR_EDGE                    0x01000000ULL
+#define P4_CCCR_THRESHOLD_MASK          0x00f00000ULL
 #define P4_CCCR_THRESHOLD_SHIFT         20
-#define P4_CCCR_COMPLEMENT              0x00080000U
-#define P4_CCCR_COMPARE                 0x00040000U
-#define P4_CCCR_ESCR_SELECT_MASK        0x0000e000U
+#define P4_CCCR_COMPLEMENT              0x00080000ULL
+#define P4_CCCR_COMPARE                 0x00040000ULL
+#define P4_CCCR_ESCR_SELECT_MASK        0x0000e000ULL
 #define P4_CCCR_ESCR_SELECT_SHIFT       13
-#define P4_CCCR_ENABLE                  0x00001000U
-#define P4_CCCR_THREAD_SINGLE           0x00010000U
-#define P4_CCCR_THREAD_BOTH             0x00020000U
-#define P4_CCCR_THREAD_ANY              0x00030000U
-#define P4_CCCR_RESERVED                0x00000fffU
+#define P4_CCCR_ENABLE                  0x00001000ULL
+#define P4_CCCR_THREAD_SINGLE           0x00010000ULL
+#define P4_CCCR_THREAD_BOTH             0x00020000ULL
+#define P4_CCCR_THREAD_ANY              0x00030000ULL
+#define P4_CCCR_RESERVED                0x00000fffULL
 
 #define P4_CCCR_THRESHOLD(v)            ((v) << P4_CCCR_THRESHOLD_SHIFT)
 #define P4_CCCR_ESEL(v)                 ((v) << P4_CCCR_ESCR_SELECT_SHIFT)
 
 #define P4_GEN_ESCR_EMASK(class, name, bit)     \
-        class##__##name = ((1 << bit) << P4_ESCR_EVENTMASK_SHIFT)
+        class##__##name = ((1ULL << bit) << P4_ESCR_EVENTMASK_SHIFT)
 #define P4_ESCR_EMASK_BIT(class, name)          class##__##name
 
 /*
@@ -107,7 +107,7 @@
  * P4_PEBS_CONFIG_MASK and related bits on
  * modification.)
  */
-#define P4_CONFIG_ALIASABLE             (1 << 9)
+#define P4_CONFIG_ALIASABLE             (1ULL << 9)
 
 /*
  * The bits we allow to pass for RAW events
@@ -784,17 +784,17 @@ enum P4_ESCR_EMASKS {
  * Note we have UOP and PEBS bits reserved for now
  * just in case if we will need them once
  */
-#define P4_PEBS_CONFIG_ENABLE           (1 << 7)
-#define P4_PEBS_CONFIG_UOP_TAG          (1 << 8)
-#define P4_PEBS_CONFIG_METRIC_MASK      0x3f
-#define P4_PEBS_CONFIG_MASK             0xff
+#define P4_PEBS_CONFIG_ENABLE           (1ULL << 7)
+#define P4_PEBS_CONFIG_UOP_TAG          (1ULL << 8)
+#define P4_PEBS_CONFIG_METRIC_MASK      0x3FLL
+#define P4_PEBS_CONFIG_MASK             0xFFLL
 
 /*
  * mem: Only counters MSR_IQ_COUNTER4 (16) and
  * MSR_IQ_COUNTER5 (17) are allowed for PEBS sampling
  */
-#define P4_PEBS_ENABLE                  0x02000000U
-#define P4_PEBS_ENABLE_UOP_TAG          0x01000000U
+#define P4_PEBS_ENABLE                  0x02000000ULL
+#define P4_PEBS_ENABLE_UOP_TAG          0x01000000ULL
 
 #define p4_config_unpack_metric(v)      (((u64)(v)) & P4_PEBS_CONFIG_METRIC_MASK)
 #define p4_config_unpack_pebs(v)        (((u64)(v)) & P4_PEBS_CONFIG_MASK)
diff --git a/arch/x86/include/asm/uprobes.h b/arch/x86/include/asm/uprobes.h
index 8ff8be7835ab..6e5197910fd8 100644
--- a/arch/x86/include/asm/uprobes.h
+++ b/arch/x86/include/asm/uprobes.h
@@ -55,4 +55,5 @@ extern int  arch_uprobe_post_xol(struct arch_uprobe *aup, struct pt_regs *regs);
 extern bool arch_uprobe_xol_was_trapped(struct task_struct *tsk);
 extern int  arch_uprobe_exception_notify(struct notifier_block *self, unsigned long val, void *data);
 extern void arch_uprobe_abort_xol(struct arch_uprobe *aup, struct pt_regs *regs);
+extern unsigned long arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs);
 #endif  /* _ASM_UPROBES_H */
diff --git a/arch/x86/include/uapi/asm/msr-index.h b/arch/x86/include/uapi/asm/msr-index.h
index 7a060f4b411f..b5757885d7a4 100644
--- a/arch/x86/include/uapi/asm/msr-index.h
+++ b/arch/x86/include/uapi/asm/msr-index.h
@@ -72,6 +72,7 @@
 #define MSR_IA32_PEBS_ENABLE            0x000003f1
 #define MSR_IA32_DS_AREA                0x00000600
 #define MSR_IA32_PERF_CAPABILITIES      0x00000345
+#define MSR_PEBS_LD_LAT_THRESHOLD       0x000003f6
 
 #define MSR_MTRRfix64K_00000            0x00000250
 #define MSR_MTRRfix16K_80000            0x00000258
@@ -195,6 +196,10 @@
 #define MSR_AMD64_IBSBRTARGET           0xc001103b
 #define MSR_AMD64_IBS_REG_COUNT_MAX     8 /* includes MSR_AMD64_IBSBRTARGET */
 
+/* Fam 16h MSRs */
+#define MSR_F16H_L2I_PERF_CTL           0xc0010230
+#define MSR_F16H_L2I_PERF_CTR           0xc0010231
+
 /* Fam 15h MSRs */
 #define MSR_F15H_PERF_CTL               0xc0010200
 #define MSR_F15H_PERF_CTR               0xc0010201
diff --git a/arch/x86/kernel/cpu/Makefile b/arch/x86/kernel/cpu/Makefile
index c9496313843b..deef0399fc78 100644
--- a/arch/x86/kernel/cpu/Makefile
+++ b/arch/x86/kernel/cpu/Makefile
@@ -31,7 +31,7 @@ obj-$(CONFIG_CPU_SUP_UMC_32)		+= umc.o
 obj-$(CONFIG_PERF_EVENTS)               += perf_event.o
 
 ifdef CONFIG_PERF_EVENTS
-obj-$(CONFIG_CPU_SUP_AMD)               += perf_event_amd.o
+obj-$(CONFIG_CPU_SUP_AMD)               += perf_event_amd.o perf_event_amd_uncore.o
 obj-$(CONFIG_CPU_SUP_INTEL)             += perf_event_p6.o perf_event_knc.o perf_event_p4.o
 obj-$(CONFIG_CPU_SUP_INTEL)             += perf_event_intel_lbr.o perf_event_intel_ds.o perf_event_intel.o
 obj-$(CONFIG_CPU_SUP_INTEL)             += perf_event_intel_uncore.o
diff --git a/arch/x86/kernel/cpu/perf_event.c b/arch/x86/kernel/cpu/perf_event.c
index bf0f01aea994..1025f3c99d20 100644
--- a/arch/x86/kernel/cpu/perf_event.c
+++ b/arch/x86/kernel/cpu/perf_event.c
@@ -180,8 +180,9 @@ static void release_pmc_hardware(void) {}
 
 static bool check_hw_exists(void)
 {
-        u64 val, val_new = ~0;
-        int i, reg, ret = 0;
+        u64 val, val_fail, val_new= ~0;
+        int i, reg, reg_fail, ret = 0;
+        int bios_fail = 0;
 
         /*
          * Check to see if the BIOS enabled any of the counters, if so
@@ -192,8 +193,11 @@ static bool check_hw_exists(void)
                 ret = rdmsrl_safe(reg, &val);
                 if (ret)
                         goto msr_fail;
-                if (val & ARCH_PERFMON_EVENTSEL_ENABLE)
-                        goto bios_fail;
+                if (val & ARCH_PERFMON_EVENTSEL_ENABLE) {
+                        bios_fail = 1;
+                        val_fail = val;
+                        reg_fail = reg;
+                }
         }
 
         if (x86_pmu.num_counters_fixed) {
@@ -202,8 +206,11 @@ static bool check_hw_exists(void)
                 if (ret)
                         goto msr_fail;
                 for (i = 0; i < x86_pmu.num_counters_fixed; i++) {
-                        if (val & (0x03 << i*4))
-                                goto bios_fail;
+                        if (val & (0x03 << i*4)) {
+                                bios_fail = 1;
+                                val_fail = val;
+                                reg_fail = reg;
+                        }
                 }
         }
 
@@ -221,14 +228,13 @@ static bool check_hw_exists(void)
         if (ret || val != val_new)
                 goto msr_fail;
 
-        return true;
-
-bios_fail:
         /*
          * We still allow the PMU driver to operate:
          */
-        printk(KERN_CONT "Broken BIOS detected, complain to your hardware vendor.\n");
-        printk(KERN_ERR FW_BUG "the BIOS has corrupted hw-PMU resources (MSR %x is %Lx)\n", reg, val);
+        if (bios_fail) {
+                printk(KERN_CONT "Broken BIOS detected, complain to your hardware vendor.\n");
+                printk(KERN_ERR FW_BUG "the BIOS has corrupted hw-PMU resources (MSR %x is %Lx)\n", reg_fail, val_fail);
+        }
 
         return true;
 
@@ -1316,9 +1322,16 @@ static struct attribute_group x86_pmu_format_group = {
  */
 static void __init filter_events(struct attribute **attrs)
 {
+        struct device_attribute *d;
+        struct perf_pmu_events_attr *pmu_attr;
         int i, j;
 
         for (i = 0; attrs[i]; i++) {
+                d = (struct device_attribute *)attrs[i];
+                pmu_attr = container_of(d, struct perf_pmu_events_attr, attr);
+                /* str trumps id */
+                if (pmu_attr->event_str)
+                        continue;
                 if (x86_pmu.event_map(i))
                         continue;
 
@@ -1330,22 +1343,45 @@ static void __init filter_events(struct attribute **attrs)
         }
 }
 
-static ssize_t events_sysfs_show(struct device *dev, struct device_attribute *attr,
+/* Merge two pointer arrays */
+static __init struct attribute **merge_attr(struct attribute **a, struct attribute **b)
+{
+        struct attribute **new;
+        int j, i;
+
+        for (j = 0; a[j]; j++)
+                ;
+        for (i = 0; b[i]; i++)
+                j++;
+        j++;
+
+        new = kmalloc(sizeof(struct attribute *) * j, GFP_KERNEL);
+        if (!new)
+                return NULL;
+
+        j = 0;
+        for (i = 0; a[i]; i++)
+                new[j++] = a[i];
+        for (i = 0; b[i]; i++)
+                new[j++] = b[i];
+        new[j] = NULL;
+
+        return new;
+}
+
+ssize_t events_sysfs_show(struct device *dev, struct device_attribute *attr,
                           char *page)
 {
         struct perf_pmu_events_attr *pmu_attr = \
                 container_of(attr, struct perf_pmu_events_attr, attr);
-
         u64 config = x86_pmu.event_map(pmu_attr->id);
-        return x86_pmu.events_sysfs_show(page, config);
-}
 
-#define EVENT_VAR(_id)  event_attr_##_id
-#define EVENT_PTR(_id) &event_attr_##_id.attr.attr
+        /* string trumps id */
+        if (pmu_attr->event_str)
+                return sprintf(page, "%s", pmu_attr->event_str);
 
-#define EVENT_ATTR(_name, _id)                                          \
-        PMU_EVENT_ATTR(_name, EVENT_VAR(_id), PERF_COUNT_HW_##_id,      \
-                        events_sysfs_show)
+        return x86_pmu.events_sysfs_show(page, config);
+}
 
 EVENT_ATTR(cpu-cycles,                  CPU_CYCLES              );
 EVENT_ATTR(instructions,                INSTRUCTIONS            );
@@ -1459,16 +1495,27 @@ static int __init init_hw_perf_events(void)
 
         unconstrained = (struct event_constraint)
                 __EVENT_CONSTRAINT(0, (1ULL << x86_pmu.num_counters) - 1,
-                                   0, x86_pmu.num_counters, 0);
+                                   0, x86_pmu.num_counters, 0, 0);
 
         x86_pmu.attr_rdpmc = 1; /* enable userspace RDPMC usage by default */
         x86_pmu_format_group.attrs = x86_pmu.format_attrs;
 
+        if (x86_pmu.event_attrs)
+                x86_pmu_events_group.attrs = x86_pmu.event_attrs;
+
         if (!x86_pmu.events_sysfs_show)
                 x86_pmu_events_group.attrs = &empty_attrs;
         else
                 filter_events(x86_pmu_events_group.attrs);
 
+        if (x86_pmu.cpu_events) {
+                struct attribute **tmp;
+
+                tmp = merge_attr(x86_pmu_events_group.attrs, x86_pmu.cpu_events);
+                if (!WARN_ON(!tmp))
+                        x86_pmu_events_group.attrs = tmp;
+        }
+
         pr_info("... version:                %d\n",     x86_pmu.version);
         pr_info("... bit width:              %d\n",     x86_pmu.cntval_bits);
         pr_info("... generic registers:      %d\n",     x86_pmu.num_counters);
diff --git a/arch/x86/kernel/cpu/perf_event.h b/arch/x86/kernel/cpu/perf_event.h
index 7f5c75c2afdd..ba9aadfa683b 100644
--- a/arch/x86/kernel/cpu/perf_event.h
+++ b/arch/x86/kernel/cpu/perf_event.h
@@ -46,6 +46,7 @@ enum extra_reg_type {
         EXTRA_REG_RSP_0 = 0,    /* offcore_response_0 */
         EXTRA_REG_RSP_1 = 1,    /* offcore_response_1 */
         EXTRA_REG_LBR   = 2,    /* lbr_select */
+        EXTRA_REG_LDLAT = 3,    /* ld_lat_threshold */
 
         EXTRA_REG_MAX           /* number of entries needed */
 };
@@ -59,7 +60,13 @@ struct event_constraint {
         u64     cmask;
         int     weight;
         int     overlap;
+        int     flags;
 };
+/*
+ * struct event_constraint flags
+ */
+#define PERF_X86_EVENT_PEBS_LDLAT       0x1 /* ld+ldlat data address sampling */
+#define PERF_X86_EVENT_PEBS_ST          0x2 /* st data address sampling */
 
 struct amd_nb {
         int nb_id;  /* NorthBridge id */
@@ -170,16 +177,17 @@ struct cpu_hw_events {
         void                            *kfree_on_online;
 };
 
-#define __EVENT_CONSTRAINT(c, n, m, w, o) {\
+#define __EVENT_CONSTRAINT(c, n, m, w, o, f) {\
         { .idxmsk64 = (n) },            \
         .code = (c),                    \
         .cmask = (m),                   \
         .weight = (w),                  \
         .overlap = (o),                 \
+        .flags = f,                     \
 }
 
 #define EVENT_CONSTRAINT(c, n, m)       \
-        __EVENT_CONSTRAINT(c, n, m, HWEIGHT(n), 0)
+        __EVENT_CONSTRAINT(c, n, m, HWEIGHT(n), 0, 0)
 
 /*
  * The overlap flag marks event constraints with overlapping counter
@@ -203,7 +211,7 @@ struct cpu_hw_events {
  * and its counter masks must be kept at a minimum.
  */
 #define EVENT_CONSTRAINT_OVERLAP(c, n, m)       \
-        __EVENT_CONSTRAINT(c, n, m, HWEIGHT(n), 1)
+        __EVENT_CONSTRAINT(c, n, m, HWEIGHT(n), 1, 0)
 
 /*
  * Constraint on the Event code.
@@ -231,6 +239,14 @@ struct cpu_hw_events {
 #define INTEL_UEVENT_CONSTRAINT(c, n)   \
         EVENT_CONSTRAINT(c, n, INTEL_ARCH_EVENT_MASK)
 
+#define INTEL_PLD_CONSTRAINT(c, n)      \
+        __EVENT_CONSTRAINT(c, n, INTEL_ARCH_EVENT_MASK, \
+                           HWEIGHT(n), 0, PERF_X86_EVENT_PEBS_LDLAT)
+
+#define INTEL_PST_CONSTRAINT(c, n)      \
+        __EVENT_CONSTRAINT(c, n, INTEL_ARCH_EVENT_MASK, \
+                          HWEIGHT(n), 0, PERF_X86_EVENT_PEBS_ST)
+
 #define EVENT_CONSTRAINT_END            \
         EVENT_CONSTRAINT(0, 0, 0)
 
@@ -260,12 +276,22 @@ struct extra_reg {
         .msr = (ms),            \
         .config_mask = (m),     \
         .valid_mask = (vm),     \
-        .idx = EXTRA_REG_##i    \
+        .idx = EXTRA_REG_##i,   \
         }
 
 #define INTEL_EVENT_EXTRA_REG(event, msr, vm, idx)      \
         EVENT_EXTRA_REG(event, msr, ARCH_PERFMON_EVENTSEL_EVENT, vm, idx)
 
+#define INTEL_UEVENT_EXTRA_REG(event, msr, vm, idx) \
+        EVENT_EXTRA_REG(event, msr, ARCH_PERFMON_EVENTSEL_EVENT | \
+                        ARCH_PERFMON_EVENTSEL_UMASK, vm, idx)
+
+#define INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(c) \
+        INTEL_UEVENT_EXTRA_REG(c, \
+                               MSR_PEBS_LD_LAT_THRESHOLD, \
+                               0xffff, \
+                               LDLAT)
+
 #define EVENT_EXTRA_END EVENT_EXTRA_REG(0, 0, 0, 0, RSP_0)
 
 union perf_capabilities {
@@ -355,8 +381,10 @@ struct x86_pmu {
          */
         int             attr_rdpmc;
         struct attribute **format_attrs;
+        struct attribute **event_attrs;
 
         ssize_t         (*events_sysfs_show)(char *page, u64 config);
+        struct attribute **cpu_events;
 
         /*
          * CPU Hotplug hooks
@@ -421,6 +449,23 @@ do {									\
 #define ERF_NO_HT_SHARING       1
 #define ERF_HAS_RSP_1           2
 
+#define EVENT_VAR(_id)  event_attr_##_id
+#define EVENT_PTR(_id) &event_attr_##_id.attr.attr
+
+#define EVENT_ATTR(_name, _id)                                          \
+static struct perf_pmu_events_attr EVENT_VAR(_id) = {                   \
+        .attr           = __ATTR(_name, 0444, events_sysfs_show, NULL), \
+        .id             = PERF_COUNT_HW_##_id,                          \
+        .event_str      = NULL,                                         \
+};
+
+#define EVENT_ATTR_STR(_name, v, str)                                   \
+static struct perf_pmu_events_attr event_attr_##v = {                   \
+        .attr           = __ATTR(_name, 0444, events_sysfs_show, NULL), \
+        .id             = 0,                                            \
+        .event_str      = str,                                          \
+};
+
 extern struct x86_pmu x86_pmu __read_mostly;
 
 DECLARE_PER_CPU(struct cpu_hw_events, cpu_hw_events);
@@ -628,6 +673,9 @@ int p6_pmu_init(void);
 
 int knc_pmu_init(void);
 
+ssize_t events_sysfs_show(struct device *dev, struct device_attribute *attr,
+                          char *page);
+
 #else /* CONFIG_CPU_SUP_INTEL */
 
 static inline void reserve_ds_buffers(void)
diff --git a/arch/x86/kernel/cpu/perf_event_amd.c b/arch/x86/kernel/cpu/perf_event_amd.c
index dfdab42aed27..7e28d9467bb4 100644
--- a/arch/x86/kernel/cpu/perf_event_amd.c
+++ b/arch/x86/kernel/cpu/perf_event_amd.c
@@ -132,14 +132,11 @@ static u64 amd_pmu_event_map(int hw_event)
         return amd_perfmon_event_map[hw_event];
 }
 
-static struct event_constraint *amd_nb_event_constraint;
-
 /*
  * Previously calculated offsets
  */
 static unsigned int event_offsets[X86_PMC_IDX_MAX] __read_mostly;
 static unsigned int count_offsets[X86_PMC_IDX_MAX] __read_mostly;
-static unsigned int rdpmc_indexes[X86_PMC_IDX_MAX] __read_mostly;
 
 /*
  * Legacy CPUs:
@@ -147,14 +144,10 @@ static unsigned int rdpmc_indexes[X86_PMC_IDX_MAX] __read_mostly;
  *
  * CPUs with core performance counter extensions:
  *   6 counters starting at 0xc0010200 each offset by 2
- *
- * CPUs with north bridge performance counter extensions:
- *   4 additional counters starting at 0xc0010240 each offset by 2
- *   (indexed right above either one of the above core counters)
  */
 static inline int amd_pmu_addr_offset(int index, bool eventsel)
 {
-        int offset, first, base;
+        int offset;
 
         if (!index)
                 return index;
@@ -167,23 +160,7 @@ static inline int amd_pmu_addr_offset(int index, bool eventsel)
         if (offset)
                 return offset;
 
-        if (amd_nb_event_constraint &&
-            test_bit(index, amd_nb_event_constraint->idxmsk)) {
-                /*
-                 * calculate the offset of NB counters with respect to
-                 * base eventsel or perfctr
-                 */
-
-                first = find_first_bit(amd_nb_event_constraint->idxmsk,
-                                       X86_PMC_IDX_MAX);
-
-                if (eventsel)
-                        base = MSR_F15H_NB_PERF_CTL - x86_pmu.eventsel;
-                else
-                        base = MSR_F15H_NB_PERF_CTR - x86_pmu.perfctr;
-
-                offset = base + ((index - first) << 1);
-        } else if (!cpu_has_perfctr_core)
+        if (!cpu_has_perfctr_core)
                 offset = index;
         else
                 offset = index << 1;
@@ -196,36 +173,6 @@ static inline int amd_pmu_addr_offset(int index, bool eventsel)
         return offset;
 }
 
-static inline int amd_pmu_rdpmc_index(int index)
-{
-        int ret, first;
-
-        if (!index)
-                return index;
-
-        ret = rdpmc_indexes[index];
-
-        if (ret)
-                return ret;
-
-        if (amd_nb_event_constraint &&
-            test_bit(index, amd_nb_event_constraint->idxmsk)) {
-                /*
-                 * according to the mnual, ECX value of the NB counters is
-                 * the index of the NB counter (0, 1, 2 or 3) plus 6
-                 */
-
-                first = find_first_bit(amd_nb_event_constraint->idxmsk,
-                                       X86_PMC_IDX_MAX);
-                ret = index - first + 6;
-        } else
-                ret = index;
-
-        rdpmc_indexes[index] = ret;
-
-        return ret;
-}
-
 static int amd_core_hw_config(struct perf_event *event)
 {
         if (event->attr.exclude_host && event->attr.exclude_guest)
@@ -245,34 +192,6 @@ static int amd_core_hw_config(struct perf_event *event)
 }
 
 /*
- * NB counters do not support the following event select bits:
- *   Host/Guest only
- *   Counter mask
- *   Invert counter mask
- *   Edge detect
- *   OS/User mode
- */
-static int amd_nb_hw_config(struct perf_event *event)
-{
-        /* for NB, we only allow system wide counting mode */
-        if (is_sampling_event(event) || event->attach_state & PERF_ATTACH_TASK)
-                return -EINVAL;
-
-        if (event->attr.exclude_user || event->attr.exclude_kernel ||
-            event->attr.exclude_host || event->attr.exclude_guest)
-                return -EINVAL;
-
-        event->hw.config &= ~(ARCH_PERFMON_EVENTSEL_USR |
-                              ARCH_PERFMON_EVENTSEL_OS);
-
-        if (event->hw.config & ~(AMD64_RAW_EVENT_MASK_NB |
-                                 ARCH_PERFMON_EVENTSEL_INT))
-                return -EINVAL;
-
-        return 0;
-}
-
-/*
  * AMD64 events are detected based on their event codes.
  */
 static inline unsigned int amd_get_event_code(struct hw_perf_event *hwc)
@@ -285,11 +204,6 @@ static inline int amd_is_nb_event(struct hw_perf_event *hwc)
         return (hwc->config & 0xe0) == 0xe0;
 }
 
-static inline int amd_is_perfctr_nb_event(struct hw_perf_event *hwc)
-{
-        return amd_nb_event_constraint && amd_is_nb_event(hwc);
-}
-
 static inline int amd_has_nb(struct cpu_hw_events *cpuc)
 {
         struct amd_nb *nb = cpuc->amd_nb;
@@ -315,9 +229,6 @@ static int amd_pmu_hw_config(struct perf_event *event)
         if (event->attr.type == PERF_TYPE_RAW)
                 event->hw.config |= event->attr.config & AMD64_RAW_EVENT_MASK;
 
-        if (amd_is_perfctr_nb_event(&event->hw))
-                return amd_nb_hw_config(event);
-
         return amd_core_hw_config(event);
 }
 
@@ -341,19 +252,6 @@ static void __amd_put_nb_event_constraints(struct cpu_hw_events *cpuc,
         }
 }
 
-static void amd_nb_interrupt_hw_config(struct hw_perf_event *hwc)
-{
-        int core_id = cpu_data(smp_processor_id()).cpu_core_id;
-
-        /* deliver interrupts only to this core */
-        if (hwc->config & ARCH_PERFMON_EVENTSEL_INT) {
-                hwc->config |= AMD64_EVENTSEL_INT_CORE_ENABLE;
-                hwc->config &= ~AMD64_EVENTSEL_INT_CORE_SEL_MASK;
-                hwc->config |= (u64)(core_id) <<
-                        AMD64_EVENTSEL_INT_CORE_SEL_SHIFT;
-        }
-}
-
  /*
   * AMD64 NorthBridge events need special treatment because
   * counter access needs to be synchronized across all cores
@@ -441,9 +339,6 @@ __amd_get_nb_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *ev
         if (new == -1)
                 return &emptyconstraint;
 
-        if (amd_is_perfctr_nb_event(hwc))
-                amd_nb_interrupt_hw_config(hwc);
-
         return &nb->event_constraints[new];
 }
 
@@ -543,8 +438,7 @@ amd_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event)
         if (!(amd_has_nb(cpuc) && amd_is_nb_event(&event->hw)))
                 return &unconstrained;
 
-        return __amd_get_nb_event_constraints(cpuc, event,
-                                              amd_nb_event_constraint);
+        return __amd_get_nb_event_constraints(cpuc, event, NULL);
 }
 
 static void amd_put_event_constraints(struct cpu_hw_events *cpuc,
@@ -643,9 +537,6 @@ static struct event_constraint amd_f15_PMC30 = EVENT_CONSTRAINT_OVERLAP(0, 0x09,
 static struct event_constraint amd_f15_PMC50 = EVENT_CONSTRAINT(0, 0x3F, 0);
 static struct event_constraint amd_f15_PMC53 = EVENT_CONSTRAINT(0, 0x38, 0);
 
-static struct event_constraint amd_NBPMC96 = EVENT_CONSTRAINT(0, 0x3C0, 0);
-static struct event_constraint amd_NBPMC74 = EVENT_CONSTRAINT(0, 0xF0, 0);
-
 static struct event_constraint *
 amd_get_event_constraints_f15h(struct cpu_hw_events *cpuc, struct perf_event *event)
 {
@@ -711,8 +602,8 @@ amd_get_event_constraints_f15h(struct cpu_hw_events *cpuc, struct perf_event *ev
                         return &amd_f15_PMC20;
                 }
         case AMD_EVENT_NB:
-                return __amd_get_nb_event_constraints(cpuc, event,
-                                                      amd_nb_event_constraint);
+                /* moved to perf_event_amd_uncore.c */
+                return &emptyconstraint;
         default:
                 return &emptyconstraint;
         }
@@ -738,7 +629,6 @@ static __initconst const struct x86_pmu amd_pmu = {
         .eventsel               = MSR_K7_EVNTSEL0,
         .perfctr                = MSR_K7_PERFCTR0,
         .addr_offset            = amd_pmu_addr_offset,
-        .rdpmc_index            = amd_pmu_rdpmc_index,
         .event_map              = amd_pmu_event_map,
         .max_events             = ARRAY_SIZE(amd_perfmon_event_map),
         .num_counters           = AMD64_NUM_COUNTERS,
@@ -790,23 +680,6 @@ static int setup_perfctr_core(void)
         return 0;
 }
 
-static int setup_perfctr_nb(void)
-{
-        if (!cpu_has_perfctr_nb)
-                return -ENODEV;
-
-        x86_pmu.num_counters += AMD64_NUM_COUNTERS_NB;
-
-        if (cpu_has_perfctr_core)
-                amd_nb_event_constraint = &amd_NBPMC96;
-        else
-                amd_nb_event_constraint = &amd_NBPMC74;
-
-        printk(KERN_INFO "perf: AMD northbridge performance counters detected\n");
-
-        return 0;
-}
-
 __init int amd_pmu_init(void)
 {
         /* Performance-monitoring supported from K7 and later: */
@@ -817,7 +690,6 @@ __init int amd_pmu_init(void)
 
         setup_event_constraints();
         setup_perfctr_core();
-        setup_perfctr_nb();
 
         /* Events are common for all AMDs */
         memcpy(hw_cache_event_ids, amd_hw_cache_event_ids,
diff --git a/arch/x86/kernel/cpu/perf_event_amd_uncore.c b/arch/x86/kernel/cpu/perf_event_amd_uncore.c
new file mode 100644
index 000000000000..c0c661adf03e
--- /dev/null
+++ b/arch/x86/kernel/cpu/perf_event_amd_uncore.c
@@ -0,0 +1,547 @@
+/*
+ * Copyright (C) 2013 Advanced Micro Devices, Inc.
+ *
+ * Author: Jacob Shin <jacob.shin@amd.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/perf_event.h>
+#include <linux/percpu.h>
+#include <linux/types.h>
+#include <linux/slab.h>
+#include <linux/init.h>
+#include <linux/cpu.h>
+#include <linux/cpumask.h>
+
+#include <asm/cpufeature.h>
+#include <asm/perf_event.h>
+#include <asm/msr.h>
+
+#define NUM_COUNTERS_NB         4
+#define NUM_COUNTERS_L2         4
+#define MAX_COUNTERS            NUM_COUNTERS_NB
+
+#define RDPMC_BASE_NB           6
+#define RDPMC_BASE_L2           10
+
+#define COUNTER_SHIFT           16
+
+struct amd_uncore {
+        int id;
+        int refcnt;
+        int cpu;
+        int num_counters;
+        int rdpmc_base;
+        u32 msr_base;
+        cpumask_t *active_mask;
+        struct pmu *pmu;
+        struct perf_event *events[MAX_COUNTERS];
+        struct amd_uncore *free_when_cpu_online;
+};
+
+static struct amd_uncore * __percpu *amd_uncore_nb;
+static struct amd_uncore * __percpu *amd_uncore_l2;
+
+static struct pmu amd_nb_pmu;
+static struct pmu amd_l2_pmu;
+
+static cpumask_t amd_nb_active_mask;
+static cpumask_t amd_l2_active_mask;
+
+static bool is_nb_event(struct perf_event *event)
+{
+        return event->pmu->type == amd_nb_pmu.type;
+}
+
+static bool is_l2_event(struct perf_event *event)
+{
+        return event->pmu->type == amd_l2_pmu.type;
+}
+
+static struct amd_uncore *event_to_amd_uncore(struct perf_event *event)
+{
+        if (is_nb_event(event) && amd_uncore_nb)
+                return *per_cpu_ptr(amd_uncore_nb, event->cpu);
+        else if (is_l2_event(event) && amd_uncore_l2)
+                return *per_cpu_ptr(amd_uncore_l2, event->cpu);
+
+        return NULL;
+}
+
+static void amd_uncore_read(struct perf_event *event)
+{
+        struct hw_perf_event *hwc = &event->hw;
+        u64 prev, new;
+        s64 delta;
+
+        /*
+         * since we do not enable counter overflow interrupts,
+         * we do not have to worry about prev_count changing on us
+         */
+
+        prev = local64_read(&hwc->prev_count);
+        rdpmcl(hwc->event_base_rdpmc, new);
+        local64_set(&hwc->prev_count, new);
+        delta = (new << COUNTER_SHIFT) - (prev << COUNTER_SHIFT);
+        delta >>= COUNTER_SHIFT;
+        local64_add(delta, &event->count);
+}
+
+static void amd_uncore_start(struct perf_event *event, int flags)
+{
+        struct hw_perf_event *hwc = &event->hw;
+
+        if (flags & PERF_EF_RELOAD)
+                wrmsrl(hwc->event_base, (u64)local64_read(&hwc->prev_count));
+
+        hwc->state = 0;
+        wrmsrl(hwc->config_base, (hwc->config | ARCH_PERFMON_EVENTSEL_ENABLE));
+        perf_event_update_userpage(event);
+}
+
+static void amd_uncore_stop(struct perf_event *event, int flags)
+{
+        struct hw_perf_event *hwc = &event->hw;
+
+        wrmsrl(hwc->config_base, hwc->config);
+        hwc->state |= PERF_HES_STOPPED;
+
+        if ((flags & PERF_EF_UPDATE) && !(hwc->state & PERF_HES_UPTODATE)) {
+                amd_uncore_read(event);
+                hwc->state |= PERF_HES_UPTODATE;
+        }
+}
+
+static int amd_uncore_add(struct perf_event *event, int flags)
+{
+        int i;
+        struct amd_uncore *uncore = event_to_amd_uncore(event);
+        struct hw_perf_event *hwc = &event->hw;
+
+        /* are we already assigned? */
+        if (hwc->idx != -1 && uncore->events[hwc->idx] == event)
+                goto out;
+
+        for (i = 0; i < uncore->num_counters; i++) {
+                if (uncore->events[i] == event) {
+                        hwc->idx = i;
+                        goto out;
+                }
+        }
+
+        /* if not, take the first available counter */
+        hwc->idx = -1;
+        for (i = 0; i < uncore->num_counters; i++) {
+                if (cmpxchg(&uncore->events[i], NULL, event) == NULL) {
+                        hwc->idx = i;
+                        break;
+                }
+        }
+
+out:
+        if (hwc->idx == -1)
+                return -EBUSY;
+
+        hwc->config_base = uncore->msr_base + (2 * hwc->idx);
+        hwc->event_base = uncore->msr_base + 1 + (2 * hwc->idx);
+        hwc->event_base_rdpmc = uncore->rdpmc_base + hwc->idx;
+        hwc->state = PERF_HES_UPTODATE | PERF_HES_STOPPED;
+
+        if (flags & PERF_EF_START)
+                amd_uncore_start(event, PERF_EF_RELOAD);
+
+        return 0;
+}
+
+static void amd_uncore_del(struct perf_event *event, int flags)
+{
+        int i;
+        struct amd_uncore *uncore = event_to_amd_uncore(event);
+        struct hw_perf_event *hwc = &event->hw;
+
+        amd_uncore_stop(event, PERF_EF_UPDATE);
+
+        for (i = 0; i < uncore->num_counters; i++) {
+                if (cmpxchg(&uncore->events[i], event, NULL) == event)
+                        break;
+        }
+
+        hwc->idx = -1;
+}
+
+static int amd_uncore_event_init(struct perf_event *event)
+{
+        struct amd_uncore *uncore;
+        struct hw_perf_event *hwc = &event->hw;
+
+        if (event->attr.type != event->pmu->type)
+                return -ENOENT;
+
+        /*
+         * NB and L2 counters (MSRs) are shared across all cores that share the
+         * same NB / L2 cache. Interrupts can be directed to a single target
+         * core, however, event counts generated by processes running on other
+         * cores cannot be masked out. So we do not support sampling and
+         * per-thread events.
+         */
+        if (is_sampling_event(event) || event->attach_state & PERF_ATTACH_TASK)
+                return -EINVAL;
+
+        /* NB and L2 counters do not have usr/os/guest/host bits */
+        if (event->attr.exclude_user || event->attr.exclude_kernel ||
+            event->attr.exclude_host || event->attr.exclude_guest)
+                return -EINVAL;
+
+        /* and we do not enable counter overflow interrupts */
+        hwc->config = event->attr.config & AMD64_RAW_EVENT_MASK_NB;
+        hwc->idx = -1;
+
+        if (event->cpu < 0)
+                return -EINVAL;
+
+        uncore = event_to_amd_uncore(event);
+        if (!uncore)
+                return -ENODEV;
+
+        /*
+         * since request can come in to any of the shared cores, we will remap
+         * to a single common cpu.
+         */
+        event->cpu = uncore->cpu;
+
+        return 0;
+}
+
+static ssize_t amd_uncore_attr_show_cpumask(struct device *dev,
+                                            struct device_attribute *attr,
+                                            char *buf)
+{
+        int n;
+        cpumask_t *active_mask;
+        struct pmu *pmu = dev_get_drvdata(dev);
+
+        if (pmu->type == amd_nb_pmu.type)
+                active_mask = &amd_nb_active_mask;
+        else if (pmu->type == amd_l2_pmu.type)
+                active_mask = &amd_l2_active_mask;
+        else
+                return 0;
+
+        n = cpulist_scnprintf(buf, PAGE_SIZE - 2, active_mask);
+        buf[n++] = '\n';
+        buf[n] = '\0';
+        return n;
+}
+static DEVICE_ATTR(cpumask, S_IRUGO, amd_uncore_attr_show_cpumask, NULL);
+
+static struct attribute *amd_uncore_attrs[] = {
+        &dev_attr_cpumask.attr,
+        NULL,
+};
+
+static struct attribute_group amd_uncore_attr_group = {
+        .attrs = amd_uncore_attrs,
+};
+
+PMU_FORMAT_ATTR(event, "config:0-7,32-35");
+PMU_FORMAT_ATTR(umask, "config:8-15");
+
+static struct attribute *amd_uncore_format_attr[] = {
+        &format_attr_event.attr,
+        &format_attr_umask.attr,
+        NULL,
+};
+
+static struct attribute_group amd_uncore_format_group = {
+        .name = "format",
+        .attrs = amd_uncore_format_attr,
+};
+
+static const struct attribute_group *amd_uncore_attr_groups[] = {
+        &amd_uncore_attr_group,
+        &amd_uncore_format_group,
+        NULL,
+};
+
+static struct pmu amd_nb_pmu = {
+        .attr_groups    = amd_uncore_attr_groups,
+        .name           = "amd_nb",
+        .event_init     = amd_uncore_event_init,
+        .add            = amd_uncore_add,
+        .del            = amd_uncore_del,
+        .start          = amd_uncore_start,
+        .stop           = amd_uncore_stop,
+        .read           = amd_uncore_read,
+};
+
+static struct pmu amd_l2_pmu = {
+        .attr_groups    = amd_uncore_attr_groups,
+        .name           = "amd_l2",
+        .event_init     = amd_uncore_event_init,
+        .add            = amd_uncore_add,
+        .del            = amd_uncore_del,
+        .start          = amd_uncore_start,
+        .stop           = amd_uncore_stop,
+        .read           = amd_uncore_read,
+};
+
+static struct amd_uncore * __cpuinit amd_uncore_alloc(unsigned int cpu)
+{
+        return kzalloc_node(sizeof(struct amd_uncore), GFP_KERNEL,
+                        cpu_to_node(cpu));
+}
+
+static void __cpuinit amd_uncore_cpu_up_prepare(unsigned int cpu)
+{
+        struct amd_uncore *uncore;
+
+        if (amd_uncore_nb) {
+                uncore = amd_uncore_alloc(cpu);
+                uncore->cpu = cpu;
+                uncore->num_counters = NUM_COUNTERS_NB;
+                uncore->rdpmc_base = RDPMC_BASE_NB;
+                uncore->msr_base = MSR_F15H_NB_PERF_CTL;
+                uncore->active_mask = &amd_nb_active_mask;
+                uncore->pmu = &amd_nb_pmu;
+                *per_cpu_ptr(amd_uncore_nb, cpu) = uncore;
+        }
+
+        if (amd_uncore_l2) {
+                uncore = amd_uncore_alloc(cpu);
+                uncore->cpu = cpu;
+                uncore->num_counters = NUM_COUNTERS_L2;
+                uncore->rdpmc_base = RDPMC_BASE_L2;
+                uncore->msr_base = MSR_F16H_L2I_PERF_CTL;
+                uncore->active_mask = &amd_l2_active_mask;
+                uncore->pmu = &amd_l2_pmu;
+                *per_cpu_ptr(amd_uncore_l2, cpu) = uncore;
+        }
+}
+
+static struct amd_uncore *
+__cpuinit amd_uncore_find_online_sibling(struct amd_uncore *this,
+                                         struct amd_uncore * __percpu *uncores)
+{
+        unsigned int cpu;
+        struct amd_uncore *that;
+
+        for_each_online_cpu(cpu) {
+                that = *per_cpu_ptr(uncores, cpu);
+
+                if (!that)
+                        continue;
+
+                if (this == that)
+                        continue;
+
+                if (this->id == that->id) {
+                        that->free_when_cpu_online = this;
+                        this = that;
+                        break;
+                }
+        }
+
+        this->refcnt++;
+        return this;
+}
+
+static void __cpuinit amd_uncore_cpu_starting(unsigned int cpu)
+{
+        unsigned int eax, ebx, ecx, edx;
+        struct amd_uncore *uncore;
+
+        if (amd_uncore_nb) {
+                uncore = *per_cpu_ptr(amd_uncore_nb, cpu);
+                cpuid(0x8000001e, &eax, &ebx, &ecx, &edx);
+                uncore->id = ecx & 0xff;
+
+                uncore = amd_uncore_find_online_sibling(uncore, amd_uncore_nb);
+                *per_cpu_ptr(amd_uncore_nb, cpu) = uncore;
+        }
+
+        if (amd_uncore_l2) {
+                unsigned int apicid = cpu_data(cpu).apicid;
+                unsigned int nshared;
+
+                uncore = *per_cpu_ptr(amd_uncore_l2, cpu);
+                cpuid_count(0x8000001d, 2, &eax, &ebx, &ecx, &edx);
+                nshared = ((eax >> 14) & 0xfff) + 1;
+                uncore->id = apicid - (apicid % nshared);
+
+                uncore = amd_uncore_find_online_sibling(uncore, amd_uncore_l2);
+                *per_cpu_ptr(amd_uncore_l2, cpu) = uncore;
+        }
+}
+
+static void __cpuinit uncore_online(unsigned int cpu,
+                                    struct amd_uncore * __percpu *uncores)
+{
+        struct amd_uncore *uncore = *per_cpu_ptr(uncores, cpu);
+
+        kfree(uncore->free_when_cpu_online);
+        uncore->free_when_cpu_online = NULL;
+
+        if (cpu == uncore->cpu)
+                cpumask_set_cpu(cpu, uncore->active_mask);
+}
+
+static void __cpuinit amd_uncore_cpu_online(unsigned int cpu)
+{
+        if (amd_uncore_nb)
+                uncore_online(cpu, amd_uncore_nb);
+
+        if (amd_uncore_l2)
+                uncore_online(cpu, amd_uncore_l2);
+}
+
+static void __cpuinit uncore_down_prepare(unsigned int cpu,
+                                          struct amd_uncore * __percpu *uncores)
+{
+        unsigned int i;
+        struct amd_uncore *this = *per_cpu_ptr(uncores, cpu);
+
+        if (this->cpu != cpu)
+                return;
+
+        /* this cpu is going down, migrate to a shared sibling if possible */
+        for_each_online_cpu(i) {
+                struct amd_uncore *that = *per_cpu_ptr(uncores, i);
+
+                if (cpu == i)
+                        continue;
+
+                if (this == that) {
+                        perf_pmu_migrate_context(this->pmu, cpu, i);
+                        cpumask_clear_cpu(cpu, that->active_mask);
+                        cpumask_set_cpu(i, that->active_mask);
+                        that->cpu = i;
+                        break;
+                }
+        }
+}
+
+static void __cpuinit amd_uncore_cpu_down_prepare(unsigned int cpu)
+{
+        if (amd_uncore_nb)
+                uncore_down_prepare(cpu, amd_uncore_nb);
+
+        if (amd_uncore_l2)
+                uncore_down_prepare(cpu, amd_uncore_l2);
+}
+
+static void __cpuinit uncore_dead(unsigned int cpu,
+                                  struct amd_uncore * __percpu *uncores)
+{
+        struct amd_uncore *uncore = *per_cpu_ptr(uncores, cpu);
+
+        if (cpu == uncore->cpu)
+                cpumask_clear_cpu(cpu, uncore->active_mask);
+
+        if (!--uncore->refcnt)
+                kfree(uncore);
+        *per_cpu_ptr(amd_uncore_nb, cpu) = NULL;
+}
+
+static void __cpuinit amd_uncore_cpu_dead(unsigned int cpu)
+{
+        if (amd_uncore_nb)
+                uncore_dead(cpu, amd_uncore_nb);
+
+        if (amd_uncore_l2)
+                uncore_dead(cpu, amd_uncore_l2);
+}
+
+static int __cpuinit
+amd_uncore_cpu_notifier(struct notifier_block *self, unsigned long action,
+                        void *hcpu)
+{
+        unsigned int cpu = (long)hcpu;
+
+        switch (action & ~CPU_TASKS_FROZEN) {
+        case CPU_UP_PREPARE:
+                amd_uncore_cpu_up_prepare(cpu);
+                break;
+
+        case CPU_STARTING:
+                amd_uncore_cpu_starting(cpu);
+                break;
+
+        case CPU_ONLINE:
+                amd_uncore_cpu_online(cpu);
+                break;
+
+        case CPU_DOWN_PREPARE:
+                amd_uncore_cpu_down_prepare(cpu);
+                break;
+
+        case CPU_UP_CANCELED:
+        case CPU_DEAD:
+                amd_uncore_cpu_dead(cpu);
+                break;
+
+        default:
+                break;
+        }
+
+        return NOTIFY_OK;
+}
+
+static struct notifier_block amd_uncore_cpu_notifier_block __cpuinitdata = {
+        .notifier_call  = amd_uncore_cpu_notifier,
+        .priority       = CPU_PRI_PERF + 1,
+};
+
+static void __init init_cpu_already_online(void *dummy)
+{
+        unsigned int cpu = smp_processor_id();
+
+        amd_uncore_cpu_starting(cpu);
+        amd_uncore_cpu_online(cpu);
+}
+
+static int __init amd_uncore_init(void)
+{
+        unsigned int cpu;
+        int ret = -ENODEV;
+
+        if (boot_cpu_data.x86_vendor != X86_VENDOR_AMD)
+                return -ENODEV;
+
+        if (!cpu_has_topoext)
+                return -ENODEV;
+
+        if (cpu_has_perfctr_nb) {
+                amd_uncore_nb = alloc_percpu(struct amd_uncore *);
+                perf_pmu_register(&amd_nb_pmu, amd_nb_pmu.name, -1);
+
+                printk(KERN_INFO "perf: AMD NB counters detected\n");
+                ret = 0;
+        }
+
+        if (cpu_has_perfctr_l2) {
+                amd_uncore_l2 = alloc_percpu(struct amd_uncore *);
+                perf_pmu_register(&amd_l2_pmu, amd_l2_pmu.name, -1);
+
+                printk(KERN_INFO "perf: AMD L2I counters detected\n");
+                ret = 0;
+        }
+
+        if (ret)
+                return -ENODEV;
+
+        get_online_cpus();
+        /* init cpus already online before registering for hotplug notifier */
+        for_each_online_cpu(cpu) {
+                amd_uncore_cpu_up_prepare(cpu);
+                smp_call_function_single(cpu, init_cpu_already_online, NULL, 1);
+        }
+
+        register_cpu_notifier(&amd_uncore_cpu_notifier_block);
+        put_online_cpus();
+
+        return 0;
+}
+device_initcall(amd_uncore_init);
diff --git a/arch/x86/kernel/cpu/perf_event_intel.c b/arch/x86/kernel/cpu/perf_event_intel.c
index cc45deb791b0..ffd6050a1de4 100644
--- a/arch/x86/kernel/cpu/perf_event_intel.c
+++ b/arch/x86/kernel/cpu/perf_event_intel.c
@@ -81,6 +81,7 @@ static struct event_constraint intel_nehalem_event_constraints[] __read_mostly =
 static struct extra_reg intel_nehalem_extra_regs[] __read_mostly =
 {
         INTEL_EVENT_EXTRA_REG(0xb7, MSR_OFFCORE_RSP_0, 0xffff, RSP_0),
+        INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(0x100b),
         EVENT_EXTRA_END
 };
 
@@ -108,6 +109,8 @@ static struct event_constraint intel_snb_event_constraints[] __read_mostly =
         INTEL_EVENT_CONSTRAINT(0x48, 0x4), /* L1D_PEND_MISS.PENDING */
         INTEL_UEVENT_CONSTRAINT(0x01c0, 0x2), /* INST_RETIRED.PREC_DIST */
         INTEL_EVENT_CONSTRAINT(0xcd, 0x8), /* MEM_TRANS_RETIRED.LOAD_LATENCY */
+        INTEL_UEVENT_CONSTRAINT(0x04a3, 0xf), /* CYCLE_ACTIVITY.CYCLES_NO_DISPATCH */
+        INTEL_UEVENT_CONSTRAINT(0x02a3, 0x4), /* CYCLE_ACTIVITY.CYCLES_L1D_PENDING */
         EVENT_CONSTRAINT_END
 };
 
@@ -136,6 +139,7 @@ static struct extra_reg intel_westmere_extra_regs[] __read_mostly =
 {
         INTEL_EVENT_EXTRA_REG(0xb7, MSR_OFFCORE_RSP_0, 0xffff, RSP_0),
         INTEL_EVENT_EXTRA_REG(0xbb, MSR_OFFCORE_RSP_1, 0xffff, RSP_1),
+        INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(0x100b),
         EVENT_EXTRA_END
 };
 
@@ -155,6 +159,8 @@ static struct event_constraint intel_gen_event_constraints[] __read_mostly =
 static struct extra_reg intel_snb_extra_regs[] __read_mostly = {
         INTEL_EVENT_EXTRA_REG(0xb7, MSR_OFFCORE_RSP_0, 0x3f807f8fffull, RSP_0),
         INTEL_EVENT_EXTRA_REG(0xbb, MSR_OFFCORE_RSP_1, 0x3f807f8fffull, RSP_1),
+        INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(0x01cd),
+        INTEL_UEVENT_PEBS_LDLAT_EXTRA_REG(0x01cd),
         EVENT_EXTRA_END
 };
 
@@ -164,6 +170,21 @@ static struct extra_reg intel_snbep_extra_regs[] __read_mostly = {
         EVENT_EXTRA_END
 };
 
+EVENT_ATTR_STR(mem-loads, mem_ld_nhm, "event=0x0b,umask=0x10,ldlat=3");
+EVENT_ATTR_STR(mem-loads, mem_ld_snb, "event=0xcd,umask=0x1,ldlat=3");
+EVENT_ATTR_STR(mem-stores, mem_st_snb, "event=0xcd,umask=0x2");
+
+struct attribute *nhm_events_attrs[] = {
+        EVENT_PTR(mem_ld_nhm),
+        NULL,
+};
+
+struct attribute *snb_events_attrs[] = {
+        EVENT_PTR(mem_ld_snb),
+        EVENT_PTR(mem_st_snb),
+        NULL,
+};
+
 static u64 intel_pmu_event_map(int hw_event)
 {
         return intel_perfmon_event_map[hw_event];
@@ -1398,8 +1419,11 @@ x86_get_event_constraints(struct cpu_hw_events *cpuc, struct perf_event *event)
 
         if (x86_pmu.event_constraints) {
                 for_each_event_constraint(c, x86_pmu.event_constraints) {
-                        if ((event->hw.config & c->cmask) == c->code)
+                        if ((event->hw.config & c->cmask) == c->code) {
+                                /* hw.flags zeroed at initialization */
+                                event->hw.flags |= c->flags;
                                 return c;
+                        }
                 }
         }
 
@@ -1444,6 +1468,7 @@ intel_put_shared_regs_event_constraints(struct cpu_hw_events *cpuc,
 static void intel_put_event_constraints(struct cpu_hw_events *cpuc,
                                         struct perf_event *event)
 {
+        event->hw.flags = 0;
         intel_put_shared_regs_event_constraints(cpuc, event);
 }
 
@@ -1767,6 +1792,8 @@ static void intel_pmu_flush_branch_stack(void)
 
 PMU_FORMAT_ATTR(offcore_rsp, "config1:0-63");
 
+PMU_FORMAT_ATTR(ldlat, "config1:0-15");
+
 static struct attribute *intel_arch3_formats_attr[] = {
         &format_attr_event.attr,
         &format_attr_umask.attr,
@@ -1777,6 +1804,7 @@ static struct attribute *intel_arch3_formats_attr[] = {
         &format_attr_cmask.attr,
 
         &format_attr_offcore_rsp.attr, /* XXX do NHM/WSM + SNB breakout */
+        &format_attr_ldlat.attr, /* PEBS load latency */
         NULL,
 };
 
@@ -2037,6 +2065,8 @@ __init int intel_pmu_init(void)
                 x86_pmu.enable_all = intel_pmu_nhm_enable_all;
                 x86_pmu.extra_regs = intel_nehalem_extra_regs;
 
+                x86_pmu.cpu_events = nhm_events_attrs;
+
                 /* UOPS_ISSUED.STALLED_CYCLES */
                 intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
                         X86_CONFIG(.event=0x0e, .umask=0x01, .inv=1, .cmask=1);
@@ -2080,6 +2110,8 @@ __init int intel_pmu_init(void)
                 x86_pmu.extra_regs = intel_westmere_extra_regs;
                 x86_pmu.er_flags |= ERF_HAS_RSP_1;
 
+                x86_pmu.cpu_events = nhm_events_attrs;
+
                 /* UOPS_ISSUED.STALLED_CYCLES */
                 intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
                         X86_CONFIG(.event=0x0e, .umask=0x01, .inv=1, .cmask=1);
@@ -2111,6 +2143,8 @@ __init int intel_pmu_init(void)
                 x86_pmu.er_flags |= ERF_HAS_RSP_1;
                 x86_pmu.er_flags |= ERF_NO_HT_SHARING;
 
+                x86_pmu.cpu_events = snb_events_attrs;
+
                 /* UOPS_ISSUED.ANY,c=1,i=1 to count stall cycles */
                 intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
                         X86_CONFIG(.event=0x0e, .umask=0x01, .inv=1, .cmask=1);
@@ -2140,6 +2174,8 @@ __init int intel_pmu_init(void)
                 x86_pmu.er_flags |= ERF_HAS_RSP_1;
                 x86_pmu.er_flags |= ERF_NO_HT_SHARING;
 
+                x86_pmu.cpu_events = snb_events_attrs;
+
                 /* UOPS_ISSUED.ANY,c=1,i=1 to count stall cycles */
                 intel_perfmon_event_map[PERF_COUNT_HW_STALLED_CYCLES_FRONTEND] =
                         X86_CONFIG(.event=0x0e, .umask=0x01, .inv=1, .cmask=1);
diff --git a/arch/x86/kernel/cpu/perf_event_intel_ds.c b/arch/x86/kernel/cpu/perf_event_intel_ds.c
index 26830f3af0df..60250f687052 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_ds.c
+++ b/arch/x86/kernel/cpu/perf_event_intel_ds.c
@@ -24,6 +24,130 @@ struct pebs_record_32 {
 
  */
 
+union intel_x86_pebs_dse {
+        u64 val;
+        struct {
+                unsigned int ld_dse:4;
+                unsigned int ld_stlb_miss:1;
+                unsigned int ld_locked:1;
+                unsigned int ld_reserved:26;
+        };
+        struct {
+                unsigned int st_l1d_hit:1;
+                unsigned int st_reserved1:3;
+                unsigned int st_stlb_miss:1;
+                unsigned int st_locked:1;
+                unsigned int st_reserved2:26;
+        };
+};
+
+
+/*
+ * Map PEBS Load Latency Data Source encodings to generic
+ * memory data source information
+ */
+#define P(a, b) PERF_MEM_S(a, b)
+#define OP_LH (P(OP, LOAD) | P(LVL, HIT))
+#define SNOOP_NONE_MISS (P(SNOOP, NONE) | P(SNOOP, MISS))
+
+static const u64 pebs_data_source[] = {
+        P(OP, LOAD) | P(LVL, MISS) | P(LVL, L3) | P(SNOOP, NA),/* 0x00:ukn L3 */
+        OP_LH | P(LVL, L1)  | P(SNOOP, NONE),   /* 0x01: L1 local */
+        OP_LH | P(LVL, LFB) | P(SNOOP, NONE),   /* 0x02: LFB hit */
+        OP_LH | P(LVL, L2)  | P(SNOOP, NONE),   /* 0x03: L2 hit */
+        OP_LH | P(LVL, L3)  | P(SNOOP, NONE),   /* 0x04: L3 hit */
+        OP_LH | P(LVL, L3)  | P(SNOOP, MISS),   /* 0x05: L3 hit, snoop miss */
+        OP_LH | P(LVL, L3)  | P(SNOOP, HIT),    /* 0x06: L3 hit, snoop hit */
+        OP_LH | P(LVL, L3)  | P(SNOOP, HITM),   /* 0x07: L3 hit, snoop hitm */
+        OP_LH | P(LVL, REM_CCE1) | P(SNOOP, HIT),  /* 0x08: L3 miss snoop hit */
+        OP_LH | P(LVL, REM_CCE1) | P(SNOOP, HITM), /* 0x09: L3 miss snoop hitm*/
+        OP_LH | P(LVL, LOC_RAM)  | P(SNOOP, HIT),  /* 0x0a: L3 miss, shared */
+        OP_LH | P(LVL, REM_RAM1) | P(SNOOP, HIT),  /* 0x0b: L3 miss, shared */
+        OP_LH | P(LVL, LOC_RAM)  | SNOOP_NONE_MISS,/* 0x0c: L3 miss, excl */
+        OP_LH | P(LVL, REM_RAM1) | SNOOP_NONE_MISS,/* 0x0d: L3 miss, excl */
+        OP_LH | P(LVL, IO)  | P(SNOOP, NONE), /* 0x0e: I/O */
+        OP_LH | P(LVL, UNC) | P(SNOOP, NONE), /* 0x0f: uncached */
+};
+
+static u64 precise_store_data(u64 status)
+{
+        union intel_x86_pebs_dse dse;
+        u64 val = P(OP, STORE) | P(SNOOP, NA) | P(LVL, L1) | P(TLB, L2);
+
+        dse.val = status;
+
+        /*
+         * bit 4: TLB access
+         * 1 = stored missed 2nd level TLB
+         *
+         * so it either hit the walker or the OS
+         * otherwise hit 2nd level TLB
+         */
+        if (dse.st_stlb_miss)
+                val |= P(TLB, MISS);
+        else
+                val |= P(TLB, HIT);
+
+        /*
+         * bit 0: hit L1 data cache
+         * if not set, then all we know is that
+         * it missed L1D
+         */
+        if (dse.st_l1d_hit)
+                val |= P(LVL, HIT);
+        else
+                val |= P(LVL, MISS);
+
+        /*
+         * bit 5: Locked prefix
+         */
+        if (dse.st_locked)
+                val |= P(LOCK, LOCKED);
+
+        return val;
+}
+
+static u64 load_latency_data(u64 status)
+{
+        union intel_x86_pebs_dse dse;
+        u64 val;
+        int model = boot_cpu_data.x86_model;
+        int fam = boot_cpu_data.x86;
+
+        dse.val = status;
+
+        /*
+         * use the mapping table for bit 0-3
+         */
+        val = pebs_data_source[dse.ld_dse];
+
+        /*
+         * Nehalem models do not support TLB, Lock infos
+         */
+        if (fam == 0x6 && (model == 26 || model == 30
+            || model == 31 || model == 46)) {
+                val |= P(TLB, NA) | P(LOCK, NA);
+                return val;
+        }
+        /*
+         * bit 4: TLB access
+         * 0 = did not miss 2nd level TLB
+         * 1 = missed 2nd level TLB
+         */
+        if (dse.ld_stlb_miss)
+                val |= P(TLB, MISS) | P(TLB, L2);
+        else
+                val |= P(TLB, HIT) | P(TLB, L1) | P(TLB, L2);
+
+        /*
+         * bit 5: locked prefix
+         */
+        if (dse.ld_locked)
+                val |= P(LOCK, LOCKED);
+
+        return val;
+}
+
 struct pebs_record_core {
         u64 flags, ip;
         u64 ax, bx, cx, dx;
@@ -365,7 +489,7 @@ struct event_constraint intel_atom_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_nehalem_pebs_event_constraints[] = {
-        INTEL_EVENT_CONSTRAINT(0x0b, 0xf),    /* MEM_INST_RETIRED.* */
+        INTEL_PLD_CONSTRAINT(0x100b, 0xf),      /* MEM_INST_RETIRED.* */
         INTEL_EVENT_CONSTRAINT(0x0f, 0xf),    /* MEM_UNCORE_RETIRED.* */
         INTEL_UEVENT_CONSTRAINT(0x010c, 0xf), /* MEM_STORE_RETIRED.DTLB_MISS */
         INTEL_EVENT_CONSTRAINT(0xc0, 0xf),    /* INST_RETIRED.ANY */
@@ -380,7 +504,7 @@ struct event_constraint intel_nehalem_pebs_event_constraints[] = {
 };
 
 struct event_constraint intel_westmere_pebs_event_constraints[] = {
-        INTEL_EVENT_CONSTRAINT(0x0b, 0xf),    /* MEM_INST_RETIRED.* */
+        INTEL_PLD_CONSTRAINT(0x100b, 0xf),      /* MEM_INST_RETIRED.* */
         INTEL_EVENT_CONSTRAINT(0x0f, 0xf),    /* MEM_UNCORE_RETIRED.* */
         INTEL_UEVENT_CONSTRAINT(0x010c, 0xf), /* MEM_STORE_RETIRED.DTLB_MISS */
         INTEL_EVENT_CONSTRAINT(0xc0, 0xf),    /* INSTR_RETIRED.* */
@@ -400,7 +524,8 @@ struct event_constraint intel_snb_pebs_event_constraints[] = {
         INTEL_UEVENT_CONSTRAINT(0x02c2, 0xf), /* UOPS_RETIRED.RETIRE_SLOTS */
         INTEL_EVENT_CONSTRAINT(0xc4, 0xf),    /* BR_INST_RETIRED.* */
         INTEL_EVENT_CONSTRAINT(0xc5, 0xf),    /* BR_MISP_RETIRED.* */
-        INTEL_EVENT_CONSTRAINT(0xcd, 0x8),    /* MEM_TRANS_RETIRED.* */
+        INTEL_PLD_CONSTRAINT(0x01cd, 0x8),    /* MEM_TRANS_RETIRED.LAT_ABOVE_THR */
+        INTEL_PST_CONSTRAINT(0x02cd, 0x8),    /* MEM_TRANS_RETIRED.PRECISE_STORES */
         INTEL_EVENT_CONSTRAINT(0xd0, 0xf),    /* MEM_UOP_RETIRED.* */
         INTEL_EVENT_CONSTRAINT(0xd1, 0xf),    /* MEM_LOAD_UOPS_RETIRED.* */
         INTEL_EVENT_CONSTRAINT(0xd2, 0xf),    /* MEM_LOAD_UOPS_LLC_HIT_RETIRED.* */
@@ -414,7 +539,8 @@ struct event_constraint intel_ivb_pebs_event_constraints[] = {
         INTEL_UEVENT_CONSTRAINT(0x02c2, 0xf), /* UOPS_RETIRED.RETIRE_SLOTS */
         INTEL_EVENT_CONSTRAINT(0xc4, 0xf),    /* BR_INST_RETIRED.* */
         INTEL_EVENT_CONSTRAINT(0xc5, 0xf),    /* BR_MISP_RETIRED.* */
-        INTEL_EVENT_CONSTRAINT(0xcd, 0x8),    /* MEM_TRANS_RETIRED.* */
+        INTEL_PLD_CONSTRAINT(0x01cd, 0x8),    /* MEM_TRANS_RETIRED.LAT_ABOVE_THR */
+        INTEL_PST_CONSTRAINT(0x02cd, 0x8),    /* MEM_TRANS_RETIRED.PRECISE_STORES */
         INTEL_EVENT_CONSTRAINT(0xd0, 0xf),    /* MEM_UOP_RETIRED.* */
         INTEL_EVENT_CONSTRAINT(0xd1, 0xf),    /* MEM_LOAD_UOPS_RETIRED.* */
         INTEL_EVENT_CONSTRAINT(0xd2, 0xf),    /* MEM_LOAD_UOPS_LLC_HIT_RETIRED.* */
@@ -431,8 +557,10 @@ struct event_constraint *intel_pebs_constraints(struct perf_event *event)
 
         if (x86_pmu.pebs_constraints) {
                 for_each_event_constraint(c, x86_pmu.pebs_constraints) {
-                        if ((event->hw.config & c->cmask) == c->code)
+                        if ((event->hw.config & c->cmask) == c->code) {
+                                event->hw.flags |= c->flags;
                                 return c;
+                        }
                 }
         }
 
@@ -447,6 +575,11 @@ void intel_pmu_pebs_enable(struct perf_event *event)
         hwc->config &= ~ARCH_PERFMON_EVENTSEL_INT;
 
         cpuc->pebs_enabled |= 1ULL << hwc->idx;
+
+        if (event->hw.flags & PERF_X86_EVENT_PEBS_LDLAT)
+                cpuc->pebs_enabled |= 1ULL << (hwc->idx + 32);
+        else if (event->hw.flags & PERF_X86_EVENT_PEBS_ST)
+                cpuc->pebs_enabled |= 1ULL << 63;
 }
 
 void intel_pmu_pebs_disable(struct perf_event *event)
@@ -559,20 +692,51 @@ static void __intel_pmu_pebs_event(struct perf_event *event,
                                    struct pt_regs *iregs, void *__pebs)
 {
         /*
-         * We cast to pebs_record_core since that is a subset of
-         * both formats and we don't use the other fields in this
-         * routine.
+         * We cast to pebs_record_nhm to get the load latency data
+         * if extra_reg MSR_PEBS_LD_LAT_THRESHOLD used
          */
         struct cpu_hw_events *cpuc = &__get_cpu_var(cpu_hw_events);
-        struct pebs_record_core *pebs = __pebs;
+        struct pebs_record_nhm *pebs = __pebs;
         struct perf_sample_data data;
         struct pt_regs regs;
+        u64 sample_type;
+        int fll, fst;
 
         if (!intel_pmu_save_and_restart(event))
                 return;
 
+        fll = event->hw.flags & PERF_X86_EVENT_PEBS_LDLAT;
+        fst = event->hw.flags & PERF_X86_EVENT_PEBS_ST;
+
         perf_sample_data_init(&data, 0, event->hw.last_period);
 
+        data.period = event->hw.last_period;
+        sample_type = event->attr.sample_type;
+
+        /*
+         * if PEBS-LL or PreciseStore
+         */
+        if (fll || fst) {
+                if (sample_type & PERF_SAMPLE_ADDR)
+                        data.addr = pebs->dla;
+
+                /*
+                 * Use latency for weight (only avail with PEBS-LL)
+                 */
+                if (fll && (sample_type & PERF_SAMPLE_WEIGHT))
+                        data.weight = pebs->lat;
+
+                /*
+                 * data.data_src encodes the data source
+                 */
+                if (sample_type & PERF_SAMPLE_DATA_SRC) {
+                        if (fll)
+                                data.data_src.val = load_latency_data(pebs->dse);
+                        else
+                                data.data_src.val = precise_store_data(pebs->dse);
+                }
+        }
+
         /*
          * We use the interrupt regs as a base because the PEBS record
          * does not contain a full regs set, specifically it seems to
diff --git a/arch/x86/kernel/cpu/perf_event_intel_uncore.c b/arch/x86/kernel/cpu/perf_event_intel_uncore.c
index b43200dbfe7e..d0f9e5aa2151 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_uncore.c
+++ b/arch/x86/kernel/cpu/perf_event_intel_uncore.c
@@ -17,6 +17,9 @@ static struct event_constraint constraint_fixed =
 static struct event_constraint constraint_empty =
         EVENT_CONSTRAINT(0, 0, 0);
 
+#define __BITS_VALUE(x, i, n)  ((typeof(x))(((x) >> ((i) * (n))) & \
+                                ((1ULL << (n)) - 1)))
+
 DEFINE_UNCORE_FORMAT_ATTR(event, event, "config:0-7");
 DEFINE_UNCORE_FORMAT_ATTR(event_ext, event, "config:0-7,21");
 DEFINE_UNCORE_FORMAT_ATTR(umask, umask, "config:8-15");
@@ -31,9 +34,13 @@ DEFINE_UNCORE_FORMAT_ATTR(occ_sel, occ_sel, "config:14-15");
 DEFINE_UNCORE_FORMAT_ATTR(occ_invert, occ_invert, "config:30");
 DEFINE_UNCORE_FORMAT_ATTR(occ_edge, occ_edge, "config:14-51");
 DEFINE_UNCORE_FORMAT_ATTR(filter_tid, filter_tid, "config1:0-4");
+DEFINE_UNCORE_FORMAT_ATTR(filter_link, filter_link, "config1:5-8");
 DEFINE_UNCORE_FORMAT_ATTR(filter_nid, filter_nid, "config1:10-17");
+DEFINE_UNCORE_FORMAT_ATTR(filter_nid2, filter_nid, "config1:32-47");
 DEFINE_UNCORE_FORMAT_ATTR(filter_state, filter_state, "config1:18-22");
+DEFINE_UNCORE_FORMAT_ATTR(filter_state2, filter_state, "config1:17-22");
 DEFINE_UNCORE_FORMAT_ATTR(filter_opc, filter_opc, "config1:23-31");
+DEFINE_UNCORE_FORMAT_ATTR(filter_opc2, filter_opc, "config1:52-60");
 DEFINE_UNCORE_FORMAT_ATTR(filter_band0, filter_band0, "config1:0-7");
 DEFINE_UNCORE_FORMAT_ATTR(filter_band1, filter_band1, "config1:8-15");
 DEFINE_UNCORE_FORMAT_ATTR(filter_band2, filter_band2, "config1:16-23");
@@ -110,6 +117,21 @@ static void uncore_put_constraint(struct intel_uncore_box *box, struct perf_even
         reg1->alloc = 0;
 }
 
+static u64 uncore_shared_reg_config(struct intel_uncore_box *box, int idx)
+{
+        struct intel_uncore_extra_reg *er;
+        unsigned long flags;
+        u64 config;
+
+        er = &box->shared_regs[idx];
+
+        raw_spin_lock_irqsave(&er->lock, flags);
+        config = er->config;
+        raw_spin_unlock_irqrestore(&er->lock, flags);
+
+        return config;
+}
+
 /* Sandy Bridge-EP uncore support */
 static struct intel_uncore_type snbep_uncore_cbox;
 static struct intel_uncore_type snbep_uncore_pcu;
@@ -205,7 +227,7 @@ static void snbep_uncore_msr_enable_event(struct intel_uncore_box *box, struct p
         struct hw_perf_event_extra *reg1 = &hwc->extra_reg;
 
         if (reg1->idx != EXTRA_REG_NONE)
-                wrmsrl(reg1->reg, reg1->config);
+                wrmsrl(reg1->reg, uncore_shared_reg_config(box, 0));
 
         wrmsrl(hwc->config_base, hwc->config | SNBEP_PMON_CTL_EN);
 }
@@ -226,29 +248,6 @@ static void snbep_uncore_msr_init_box(struct intel_uncore_box *box)
                 wrmsrl(msr, SNBEP_PMON_BOX_CTL_INT);
 }
 
-static int snbep_uncore_hw_config(struct intel_uncore_box *box, struct perf_event *event)
-{
-        struct hw_perf_event *hwc = &event->hw;
-        struct hw_perf_event_extra *reg1 = &hwc->extra_reg;
-
-        if (box->pmu->type == &snbep_uncore_cbox) {
-                reg1->reg = SNBEP_C0_MSR_PMON_BOX_FILTER +
-                        SNBEP_CBO_MSR_OFFSET * box->pmu->pmu_idx;
-                reg1->config = event->attr.config1 &
-                        SNBEP_CB0_MSR_PMON_BOX_FILTER_MASK;
-        } else {
-                if (box->pmu->type == &snbep_uncore_pcu) {
-                        reg1->reg = SNBEP_PCU_MSR_PMON_BOX_FILTER;
-                        reg1->config = event->attr.config1 & SNBEP_PCU_MSR_PMON_BOX_FILTER_MASK;
-                } else {
-                        return 0;
-                }
-        }
-        reg1->idx = 0;
-
-        return 0;
-}
-
 static struct attribute *snbep_uncore_formats_attr[] = {
         &format_attr_event.attr,
         &format_attr_umask.attr,
@@ -345,16 +344,16 @@ static struct attribute_group snbep_uncore_qpi_format_group = {
         .attrs = snbep_uncore_qpi_formats_attr,
 };
 
+#define SNBEP_UNCORE_MSR_OPS_COMMON_INIT()                      \
+        .init_box       = snbep_uncore_msr_init_box,            \
+        .disable_box    = snbep_uncore_msr_disable_box,         \
+        .enable_box     = snbep_uncore_msr_enable_box,          \
+        .disable_event  = snbep_uncore_msr_disable_event,       \
+        .enable_event   = snbep_uncore_msr_enable_event,        \
+        .read_counter   = uncore_msr_read_counter
+
 static struct intel_uncore_ops snbep_uncore_msr_ops = {
-        .init_box       = snbep_uncore_msr_init_box,
-        .disable_box    = snbep_uncore_msr_disable_box,
-        .enable_box     = snbep_uncore_msr_enable_box,
-        .disable_event  = snbep_uncore_msr_disable_event,
-        .enable_event   = snbep_uncore_msr_enable_event,
-        .read_counter   = uncore_msr_read_counter,
-        .get_constraint = uncore_get_constraint,
-        .put_constraint = uncore_put_constraint,
-        .hw_config      = snbep_uncore_hw_config,
+        SNBEP_UNCORE_MSR_OPS_COMMON_INIT(),
 };
 
 static struct intel_uncore_ops snbep_uncore_pci_ops = {
@@ -372,6 +371,7 @@ static struct event_constraint snbep_uncore_cbox_constraints[] = {
         UNCORE_EVENT_CONSTRAINT(0x04, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x05, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x07, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x09, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x11, 0x1),
         UNCORE_EVENT_CONSTRAINT(0x12, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x13, 0x3),
@@ -421,6 +421,14 @@ static struct event_constraint snbep_uncore_r3qpi_constraints[] = {
         UNCORE_EVENT_CONSTRAINT(0x24, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x25, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x26, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x28, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x29, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x2a, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x2b, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x2c, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x2d, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x2e, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x2f, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x30, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x31, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x32, 0x3),
@@ -428,6 +436,8 @@ static struct event_constraint snbep_uncore_r3qpi_constraints[] = {
         UNCORE_EVENT_CONSTRAINT(0x34, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x36, 0x3),
         UNCORE_EVENT_CONSTRAINT(0x37, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x38, 0x3),
+        UNCORE_EVENT_CONSTRAINT(0x39, 0x3),
         EVENT_CONSTRAINT_END
 };
 
@@ -446,6 +456,145 @@ static struct intel_uncore_type snbep_uncore_ubox = {
         .format_group   = &snbep_uncore_ubox_format_group,
 };
 
+static struct extra_reg snbep_uncore_cbox_extra_regs[] = {
+        SNBEP_CBO_EVENT_EXTRA_REG(SNBEP_CBO_PMON_CTL_TID_EN,
+                                  SNBEP_CBO_PMON_CTL_TID_EN, 0x1),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0334, 0xffff, 0x4),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0534, 0xffff, 0x4),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0934, 0xffff, 0x4),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4134, 0xffff, 0x6),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0135, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0335, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4135, 0xffff, 0xc),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4335, 0xffff, 0xc),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4435, 0xffff, 0x2),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4835, 0xffff, 0x2),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4a35, 0xffff, 0x2),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x5035, 0xffff, 0x2),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0136, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0336, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4136, 0xffff, 0xc),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4336, 0xffff, 0xc),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4436, 0xffff, 0x2),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4836, 0xffff, 0x2),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4a36, 0xffff, 0x2),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4037, 0x40ff, 0x2),
+        EVENT_EXTRA_END
+};
+
+static void snbep_cbox_put_constraint(struct intel_uncore_box *box, struct perf_event *event)
+{
+        struct hw_perf_event_extra *reg1 = &event->hw.extra_reg;
+        struct intel_uncore_extra_reg *er = &box->shared_regs[0];
+        int i;
+
+        if (uncore_box_is_fake(box))
+                return;
+
+        for (i = 0; i < 5; i++) {
+                if (reg1->alloc & (0x1 << i))
+                        atomic_sub(1 << (i * 6), &er->ref);
+        }
+        reg1->alloc = 0;
+}
+
+static struct event_constraint *
+__snbep_cbox_get_constraint(struct intel_uncore_box *box, struct perf_event *event,
+                            u64 (*cbox_filter_mask)(int fields))
+{
+        struct hw_perf_event_extra *reg1 = &event->hw.extra_reg;
+        struct intel_uncore_extra_reg *er = &box->shared_regs[0];
+        int i, alloc = 0;
+        unsigned long flags;
+        u64 mask;
+
+        if (reg1->idx == EXTRA_REG_NONE)
+                return NULL;
+
+        raw_spin_lock_irqsave(&er->lock, flags);
+        for (i = 0; i < 5; i++) {
+                if (!(reg1->idx & (0x1 << i)))
+                        continue;
+                if (!uncore_box_is_fake(box) && (reg1->alloc & (0x1 << i)))
+                        continue;
+
+                mask = cbox_filter_mask(0x1 << i);
+                if (!__BITS_VALUE(atomic_read(&er->ref), i, 6) ||
+                    !((reg1->config ^ er->config) & mask)) {
+                        atomic_add(1 << (i * 6), &er->ref);
+                        er->config &= ~mask;
+                        er->config |= reg1->config & mask;
+                        alloc |= (0x1 << i);
+                } else {
+                        break;
+                }
+        }
+        raw_spin_unlock_irqrestore(&er->lock, flags);
+        if (i < 5)
+                goto fail;
+
+        if (!uncore_box_is_fake(box))
+                reg1->alloc |= alloc;
+
+        return 0;
+fail:
+        for (; i >= 0; i--) {
+                if (alloc & (0x1 << i))
+                        atomic_sub(1 << (i * 6), &er->ref);
+        }
+        return &constraint_empty;
+}
+
+static u64 snbep_cbox_filter_mask(int fields)
+{
+        u64 mask = 0;
+
+        if (fields & 0x1)
+                mask |= SNBEP_CB0_MSR_PMON_BOX_FILTER_TID;
+        if (fields & 0x2)
+                mask |= SNBEP_CB0_MSR_PMON_BOX_FILTER_NID;
+        if (fields & 0x4)
+                mask |= SNBEP_CB0_MSR_PMON_BOX_FILTER_STATE;
+        if (fields & 0x8)
+                mask |= SNBEP_CB0_MSR_PMON_BOX_FILTER_OPC;
+
+        return mask;
+}
+
+static struct event_constraint *
+snbep_cbox_get_constraint(struct intel_uncore_box *box, struct perf_event *event)
+{
+        return __snbep_cbox_get_constraint(box, event, snbep_cbox_filter_mask);
+}
+
+static int snbep_cbox_hw_config(struct intel_uncore_box *box, struct perf_event *event)
+{
+        struct hw_perf_event_extra *reg1 = &event->hw.extra_reg;
+        struct extra_reg *er;
+        int idx = 0;
+
+        for (er = snbep_uncore_cbox_extra_regs; er->msr; er++) {
+                if (er->event != (event->hw.config & er->config_mask))
+                        continue;
+                idx |= er->idx;
+        }
+
+        if (idx) {
+                reg1->reg = SNBEP_C0_MSR_PMON_BOX_FILTER +
+                        SNBEP_CBO_MSR_OFFSET * box->pmu->pmu_idx;
+                reg1->config = event->attr.config1 & snbep_cbox_filter_mask(idx);
+                reg1->idx = idx;
+        }
+        return 0;
+}
+
+static struct intel_uncore_ops snbep_uncore_cbox_ops = {
+        SNBEP_UNCORE_MSR_OPS_COMMON_INIT(),
+        .hw_config              = snbep_cbox_hw_config,
+        .get_constraint         = snbep_cbox_get_constraint,
+        .put_constraint         = snbep_cbox_put_constraint,
+};
+
 static struct intel_uncore_type snbep_uncore_cbox = {
         .name                   = "cbox",
         .num_counters           = 4,
@@ -458,10 +607,104 @@ static struct intel_uncore_type snbep_uncore_cbox = {
         .msr_offset             = SNBEP_CBO_MSR_OFFSET,
         .num_shared_regs        = 1,
         .constraints            = snbep_uncore_cbox_constraints,
-        .ops                    = &snbep_uncore_msr_ops,
+        .ops                    = &snbep_uncore_cbox_ops,
         .format_group           = &snbep_uncore_cbox_format_group,
 };
 
+static u64 snbep_pcu_alter_er(struct perf_event *event, int new_idx, bool modify)
+{
+        struct hw_perf_event *hwc = &event->hw;
+        struct hw_perf_event_extra *reg1 = &hwc->extra_reg;
+        u64 config = reg1->config;
+
+        if (new_idx > reg1->idx)
+                config <<= 8 * (new_idx - reg1->idx);
+        else
+                config >>= 8 * (reg1->idx - new_idx);
+
+        if (modify) {
+                hwc->config += new_idx - reg1->idx;
+                reg1->config = config;
+                reg1->idx = new_idx;
+        }
+        return config;
+}
+
+static struct event_constraint *
+snbep_pcu_get_constraint(struct intel_uncore_box *box, struct perf_event *event)
+{
+        struct hw_perf_event_extra *reg1 = &event->hw.extra_reg;
+        struct intel_uncore_extra_reg *er = &box->shared_regs[0];
+        unsigned long flags;
+        int idx = reg1->idx;
+        u64 mask, config1 = reg1->config;
+        bool ok = false;
+
+        if (reg1->idx == EXTRA_REG_NONE ||
+            (!uncore_box_is_fake(box) && reg1->alloc))
+                return NULL;
+again:
+        mask = 0xff << (idx * 8);
+        raw_spin_lock_irqsave(&er->lock, flags);
+        if (!__BITS_VALUE(atomic_read(&er->ref), idx, 8) ||
+            !((config1 ^ er->config) & mask)) {
+                atomic_add(1 << (idx * 8), &er->ref);
+                er->config &= ~mask;
+                er->config |= config1 & mask;
+                ok = true;
+        }
+        raw_spin_unlock_irqrestore(&er->lock, flags);
+
+        if (!ok) {
+                idx = (idx + 1) % 4;
+                if (idx != reg1->idx) {
+                        config1 = snbep_pcu_alter_er(event, idx, false);
+                        goto again;
+                }
+                return &constraint_empty;
+        }
+
+        if (!uncore_box_is_fake(box)) {
+                if (idx != reg1->idx)
+                        snbep_pcu_alter_er(event, idx, true);
+                reg1->alloc = 1;
+        }
+        return NULL;
+}
+
+static void snbep_pcu_put_constraint(struct intel_uncore_box *box, struct perf_event *event)
+{
+        struct hw_perf_event_extra *reg1 = &event->hw.extra_reg;
+        struct intel_uncore_extra_reg *er = &box->shared_regs[0];
+
+        if (uncore_box_is_fake(box) || !reg1->alloc)
+                return;
+
+        atomic_sub(1 << (reg1->idx * 8), &er->ref);
+        reg1->alloc = 0;
+}
+
+static int snbep_pcu_hw_config(struct intel_uncore_box *box, struct perf_event *event)
+{
+        struct hw_perf_event *hwc = &event->hw;
+        struct hw_perf_event_extra *reg1 = &hwc->extra_reg;
+        int ev_sel = hwc->config & SNBEP_PMON_CTL_EV_SEL_MASK;
+
+        if (ev_sel >= 0xb && ev_sel <= 0xe) {
+                reg1->reg = SNBEP_PCU_MSR_PMON_BOX_FILTER;
+                reg1->idx = ev_sel - 0xb;
+                reg1->config = event->attr.config1 & (0xff << reg1->idx);
+        }
+        return 0;
+}
+
+static struct intel_uncore_ops snbep_uncore_pcu_ops = {
+        SNBEP_UNCORE_MSR_OPS_COMMON_INIT(),
+        .hw_config              = snbep_pcu_hw_config,
+        .get_constraint         = snbep_pcu_get_constraint,
+        .put_constraint         = snbep_pcu_put_constraint,
+};
+
 static struct intel_uncore_type snbep_uncore_pcu = {
         .name                   = "pcu",
         .num_counters           = 4,
@@ -472,7 +715,7 @@ static struct intel_uncore_type snbep_uncore_pcu = {
         .event_mask             = SNBEP_PCU_MSR_PMON_RAW_EVENT_MASK,
         .box_ctl                = SNBEP_PCU_MSR_PMON_BOX_CTL,
         .num_shared_regs        = 1,
-        .ops                    = &snbep_uncore_msr_ops,
+        .ops                    = &snbep_uncore_pcu_ops,
         .format_group           = &snbep_uncore_pcu_format_group,
 };
 
@@ -544,55 +787,63 @@ static struct intel_uncore_type snbep_uncore_r3qpi = {
         SNBEP_UNCORE_PCI_COMMON_INIT(),
 };
 
+enum {
+        SNBEP_PCI_UNCORE_HA,
+        SNBEP_PCI_UNCORE_IMC,
+        SNBEP_PCI_UNCORE_QPI,
+        SNBEP_PCI_UNCORE_R2PCIE,
+        SNBEP_PCI_UNCORE_R3QPI,
+};
+
 static struct intel_uncore_type *snbep_pci_uncores[] = {
-        &snbep_uncore_ha,
-        &snbep_uncore_imc,
-        &snbep_uncore_qpi,
-        &snbep_uncore_r2pcie,
-        &snbep_uncore_r3qpi,
+        [SNBEP_PCI_UNCORE_HA]           = &snbep_uncore_ha,
+        [SNBEP_PCI_UNCORE_IMC]          = &snbep_uncore_imc,
+        [SNBEP_PCI_UNCORE_QPI]          = &snbep_uncore_qpi,
+        [SNBEP_PCI_UNCORE_R2PCIE]       = &snbep_uncore_r2pcie,
+        [SNBEP_PCI_UNCORE_R3QPI]        = &snbep_uncore_r3qpi,
         NULL,
 };
 
 static DEFINE_PCI_DEVICE_TABLE(snbep_uncore_pci_ids) = {
         { /* Home Agent */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_HA),
-                .driver_data = (unsigned long)&snbep_uncore_ha,
+                .driver_data = SNBEP_PCI_UNCORE_HA,
         },
         { /* MC Channel 0 */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_IMC0),
-                .driver_data = (unsigned long)&snbep_uncore_imc,
+                .driver_data = SNBEP_PCI_UNCORE_IMC,
         },
         { /* MC Channel 1 */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_IMC1),
-                .driver_data = (unsigned long)&snbep_uncore_imc,
+                .driver_data = SNBEP_PCI_UNCORE_IMC,
         },
         { /* MC Channel 2 */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_IMC2),
-                .driver_data = (unsigned long)&snbep_uncore_imc,
+                .driver_data = SNBEP_PCI_UNCORE_IMC,
         },
         { /* MC Channel 3 */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_IMC3),
-                .driver_data = (unsigned long)&snbep_uncore_imc,
+                .driver_data = SNBEP_PCI_UNCORE_IMC,
         },
         { /* QPI Port 0 */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_QPI0),
-                .driver_data = (unsigned long)&snbep_uncore_qpi,
+                .driver_data = SNBEP_PCI_UNCORE_QPI,
         },
         { /* QPI Port 1 */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_QPI1),
-                .driver_data = (unsigned long)&snbep_uncore_qpi,
+                .driver_data = SNBEP_PCI_UNCORE_QPI,
         },
-        { /* P2PCIe */
+        { /* R2PCIe */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_R2PCIE),
-                .driver_data = (unsigned long)&snbep_uncore_r2pcie,
+                .driver_data = SNBEP_PCI_UNCORE_R2PCIE,
         },
         { /* R3QPI Link 0 */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_R3QPI0),
-                .driver_data = (unsigned long)&snbep_uncore_r3qpi,
+                .driver_data = SNBEP_PCI_UNCORE_R3QPI,
         },
         { /* R3QPI Link 1 */
                 PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_UNC_R3QPI1),
-                .driver_data = (unsigned long)&snbep_uncore_r3qpi,
+                .driver_data = SNBEP_PCI_UNCORE_R3QPI,
         },
         { /* end: all zeroes */ }
 };
@@ -605,7 +856,7 @@ static struct pci_driver snbep_uncore_pci_driver = {
 /*
  * build pci bus to socket mapping
  */
-static int snbep_pci2phy_map_init(void)
+static int snbep_pci2phy_map_init(int devid)
 {
         struct pci_dev *ubox_dev = NULL;
         int i, bus, nodeid;
@@ -614,9 +865,7 @@ static int snbep_pci2phy_map_init(void)
 
         while (1) {
                 /* find the UBOX device */
-                ubox_dev = pci_get_device(PCI_VENDOR_ID_INTEL,
-                                        PCI_DEVICE_ID_INTEL_JAKETOWN_UBOX,
-                                        ubox_dev);
+                ubox_dev = pci_get_device(PCI_VENDOR_ID_INTEL, devid, ubox_dev);
                 if (!ubox_dev)
                         break;
                 bus = ubox_dev->bus->number;
@@ -639,7 +888,7 @@ static int snbep_pci2phy_map_init(void)
                                 break;
                         }
                 }
-        };
+        }
 
         if (ubox_dev)
                 pci_dev_put(ubox_dev);
@@ -648,6 +897,440 @@ static int snbep_pci2phy_map_init(void)
 }
 /* end of Sandy Bridge-EP uncore support */
 
+/* IvyTown uncore support */
+static void ivt_uncore_msr_init_box(struct intel_uncore_box *box)
+{
+        unsigned msr = uncore_msr_box_ctl(box);
+        if (msr)
+                wrmsrl(msr, IVT_PMON_BOX_CTL_INT);
+}
+
+static void ivt_uncore_pci_init_box(struct intel_uncore_box *box)
+{
+        struct pci_dev *pdev = box->pci_dev;
+
+        pci_write_config_dword(pdev, SNBEP_PCI_PMON_BOX_CTL, IVT_PMON_BOX_CTL_INT);
+}
+
+#define IVT_UNCORE_MSR_OPS_COMMON_INIT()                        \
+        .init_box       = ivt_uncore_msr_init_box,              \
+        .disable_box    = snbep_uncore_msr_disable_box,         \
+        .enable_box     = snbep_uncore_msr_enable_box,          \
+        .disable_event  = snbep_uncore_msr_disable_event,       \
+        .enable_event   = snbep_uncore_msr_enable_event,        \
+        .read_counter   = uncore_msr_read_counter
+
+static struct intel_uncore_ops ivt_uncore_msr_ops = {
+        IVT_UNCORE_MSR_OPS_COMMON_INIT(),
+};
+
+static struct intel_uncore_ops ivt_uncore_pci_ops = {
+        .init_box       = ivt_uncore_pci_init_box,
+        .disable_box    = snbep_uncore_pci_disable_box,
+        .enable_box     = snbep_uncore_pci_enable_box,
+        .disable_event  = snbep_uncore_pci_disable_event,
+        .enable_event   = snbep_uncore_pci_enable_event,
+        .read_counter   = snbep_uncore_pci_read_counter,
+};
+
+#define IVT_UNCORE_PCI_COMMON_INIT()                            \
+        .perf_ctr       = SNBEP_PCI_PMON_CTR0,                  \
+        .event_ctl      = SNBEP_PCI_PMON_CTL0,                  \
+        .event_mask     = IVT_PMON_RAW_EVENT_MASK,              \
+        .box_ctl        = SNBEP_PCI_PMON_BOX_CTL,               \
+        .ops            = &ivt_uncore_pci_ops,                  \
+        .format_group   = &ivt_uncore_format_group
+
+static struct attribute *ivt_uncore_formats_attr[] = {
+        &format_attr_event.attr,
+        &format_attr_umask.attr,
+        &format_attr_edge.attr,
+        &format_attr_inv.attr,
+        &format_attr_thresh8.attr,
+        NULL,
+};
+
+static struct attribute *ivt_uncore_ubox_formats_attr[] = {
+        &format_attr_event.attr,
+        &format_attr_umask.attr,
+        &format_attr_edge.attr,
+        &format_attr_inv.attr,
+        &format_attr_thresh5.attr,
+        NULL,
+};
+
+static struct attribute *ivt_uncore_cbox_formats_attr[] = {
+        &format_attr_event.attr,
+        &format_attr_umask.attr,
+        &format_attr_edge.attr,
+        &format_attr_tid_en.attr,
+        &format_attr_thresh8.attr,
+        &format_attr_filter_tid.attr,
+        &format_attr_filter_link.attr,
+        &format_attr_filter_state2.attr,
+        &format_attr_filter_nid2.attr,
+        &format_attr_filter_opc2.attr,
+        NULL,
+};
+
+static struct attribute *ivt_uncore_pcu_formats_attr[] = {
+        &format_attr_event_ext.attr,
+        &format_attr_occ_sel.attr,
+        &format_attr_edge.attr,
+        &format_attr_thresh5.attr,
+        &format_attr_occ_invert.attr,
+        &format_attr_occ_edge.attr,
+        &format_attr_filter_band0.attr,
+        &format_attr_filter_band1.attr,
+        &format_attr_filter_band2.attr,
+        &format_attr_filter_band3.attr,
+        NULL,
+};
+
+static struct attribute *ivt_uncore_qpi_formats_attr[] = {
+        &format_attr_event_ext.attr,
+        &format_attr_umask.attr,
+        &format_attr_edge.attr,
+        &format_attr_thresh8.attr,
+        NULL,
+};
+
+static struct attribute_group ivt_uncore_format_group = {
+        .name = "format",
+        .attrs = ivt_uncore_formats_attr,
+};
+
+static struct attribute_group ivt_uncore_ubox_format_group = {
+        .name = "format",
+        .attrs = ivt_uncore_ubox_formats_attr,
+};
+
+static struct attribute_group ivt_uncore_cbox_format_group = {
+        .name = "format",
+        .attrs = ivt_uncore_cbox_formats_attr,
+};
+
+static struct attribute_group ivt_uncore_pcu_format_group = {
+        .name = "format",
+        .attrs = ivt_uncore_pcu_formats_attr,
+};
+
+static struct attribute_group ivt_uncore_qpi_format_group = {
+        .name = "format",
+        .attrs = ivt_uncore_qpi_formats_attr,
+};
+
+static struct intel_uncore_type ivt_uncore_ubox = {
+        .name           = "ubox",
+        .num_counters   = 2,
+        .num_boxes      = 1,
+        .perf_ctr_bits  = 44,
+        .fixed_ctr_bits = 48,
+        .perf_ctr       = SNBEP_U_MSR_PMON_CTR0,
+        .event_ctl      = SNBEP_U_MSR_PMON_CTL0,
+        .event_mask     = IVT_U_MSR_PMON_RAW_EVENT_MASK,
+        .fixed_ctr      = SNBEP_U_MSR_PMON_UCLK_FIXED_CTR,
+        .fixed_ctl      = SNBEP_U_MSR_PMON_UCLK_FIXED_CTL,
+        .ops            = &ivt_uncore_msr_ops,
+        .format_group   = &ivt_uncore_ubox_format_group,
+};
+
+static struct extra_reg ivt_uncore_cbox_extra_regs[] = {
+        SNBEP_CBO_EVENT_EXTRA_REG(SNBEP_CBO_PMON_CTL_TID_EN,
+                                  SNBEP_CBO_PMON_CTL_TID_EN, 0x1),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x1031, 0x10ff, 0x2),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0334, 0xffff, 0x4),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0534, 0xffff, 0x4),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0934, 0xffff, 0x4),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4134, 0xffff, 0xc),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0135, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0335, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x2135, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x2335, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4135, 0xffff, 0x18),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4335, 0xffff, 0x18),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4435, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4835, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4a35, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x5035, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x8135, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x8335, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0136, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x0336, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x2336, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x2336, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4136, 0xffff, 0x18),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4336, 0xffff, 0x18),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4436, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4836, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4a36, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x5036, 0xffff, 0x8),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x8136, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x8336, 0xffff, 0x10),
+        SNBEP_CBO_EVENT_EXTRA_REG(0x4037, 0x40ff, 0x8),
+        EVENT_EXTRA_END
+};
+
+static u64 ivt_cbox_filter_mask(int fields)
+{
+        u64 mask = 0;
+
+        if (fields & 0x1)
+                mask |= IVT_CB0_MSR_PMON_BOX_FILTER_TID;
+        if (fields & 0x2)
+                mask |= IVT_CB0_MSR_PMON_BOX_FILTER_LINK;
+        if (fields & 0x4)
+                mask |= IVT_CB0_MSR_PMON_BOX_FILTER_STATE;
+        if (fields & 0x8)
+                mask |= IVT_CB0_MSR_PMON_BOX_FILTER_NID;
+        if (fields & 0x10)
+                mask |= IVT_CB0_MSR_PMON_BOX_FILTER_OPC;
+
+        return mask;
+}
+
+static struct event_constraint *
+ivt_cbox_get_constraint(struct intel_uncore_box *box, struct perf_event *event)
+{
+        return __snbep_cbox_get_constraint(box, event, ivt_cbox_filter_mask);
+}
+
+static int ivt_cbox_hw_config(struct intel_uncore_box *box, struct perf_event *event)
+{
+        struct hw_perf_event_extra *reg1 = &event->hw.extra_reg;
+        struct extra_reg *er;
+        int idx = 0;
+
+        for (er = ivt_uncore_cbox_extra_regs; er->msr; er++) {
+                if (er->event != (event->hw.config & er->config_mask))
+                        continue;
+                idx |= er->idx;
+        }
+
+        if (idx) {
+                reg1->reg = SNBEP_C0_MSR_PMON_BOX_FILTER +
+                        SNBEP_CBO_MSR_OFFSET * box->pmu->pmu_idx;
+                reg1->config = event->attr.config1 & ivt_cbox_filter_mask(idx);
+                reg1->idx = idx;
+        }
+        return 0;
+}
+
+static void ivt_cbox_enable_event(struct intel_uncore_box *box, struct perf_event *event)
+{
+        struct hw_perf_event *hwc = &event->hw;
+        struct hw_perf_event_extra *reg1 = &hwc->extra_reg;
+
+        if (reg1->idx != EXTRA_REG_NONE) {
+                u64 filter = uncore_shared_reg_config(box, 0);
+                wrmsrl(reg1->reg, filter & 0xffffffff);
+                wrmsrl(reg1->reg + 6, filter >> 32);
+        }
+
+        wrmsrl(hwc->config_base, hwc->config | SNBEP_PMON_CTL_EN);
+}
+
+static struct intel_uncore_ops ivt_uncore_cbox_ops = {
+        .init_box               = ivt_uncore_msr_init_box,
+        .disable_box            = snbep_uncore_msr_disable_box,
+        .enable_box             = snbep_uncore_msr_enable_box,
+        .disable_event          = snbep_uncore_msr_disable_event,
+        .enable_event           = ivt_cbox_enable_event,
+        .read_counter           = uncore_msr_read_counter,
+        .hw_config              = ivt_cbox_hw_config,
+        .get_constraint         = ivt_cbox_get_constraint,
+        .put_constraint         = snbep_cbox_put_constraint,
+};
+
+static struct intel_uncore_type ivt_uncore_cbox = {
+        .name                   = "cbox",
+        .num_counters           = 4,
+        .num_boxes              = 15,
+        .perf_ctr_bits          = 44,
+        .event_ctl              = SNBEP_C0_MSR_PMON_CTL0,
+        .perf_ctr               = SNBEP_C0_MSR_PMON_CTR0,
+        .event_mask             = IVT_CBO_MSR_PMON_RAW_EVENT_MASK,
+        .box_ctl                = SNBEP_C0_MSR_PMON_BOX_CTL,
+        .msr_offset             = SNBEP_CBO_MSR_OFFSET,
+        .num_shared_regs        = 1,
+        .constraints            = snbep_uncore_cbox_constraints,
+        .ops                    = &ivt_uncore_cbox_ops,
+        .format_group           = &ivt_uncore_cbox_format_group,
+};
+
+static struct intel_uncore_ops ivt_uncore_pcu_ops = {
+        IVT_UNCORE_MSR_OPS_COMMON_INIT(),
+        .hw_config              = snbep_pcu_hw_config,
+        .get_constraint         = snbep_pcu_get_constraint,
+        .put_constraint         = snbep_pcu_put_constraint,
+};
+
+static struct intel_uncore_type ivt_uncore_pcu = {
+        .name                   = "pcu",
+        .num_counters           = 4,
+        .num_boxes              = 1,
+        .perf_ctr_bits          = 48,
+        .perf_ctr               = SNBEP_PCU_MSR_PMON_CTR0,
+        .event_ctl              = SNBEP_PCU_MSR_PMON_CTL0,
+        .event_mask             = IVT_PCU_MSR_PMON_RAW_EVENT_MASK,
+        .box_ctl                = SNBEP_PCU_MSR_PMON_BOX_CTL,
+        .num_shared_regs        = 1,
+        .ops                    = &ivt_uncore_pcu_ops,
+        .format_group           = &ivt_uncore_pcu_format_group,
+};
+
+static struct intel_uncore_type *ivt_msr_uncores[] = {
+        &ivt_uncore_ubox,
+        &ivt_uncore_cbox,
+        &ivt_uncore_pcu,
+        NULL,
+};
+
+static struct intel_uncore_type ivt_uncore_ha = {
+        .name           = "ha",
+        .num_counters   = 4,
+        .num_boxes      = 2,
+        .perf_ctr_bits  = 48,
+        IVT_UNCORE_PCI_COMMON_INIT(),
+};
+
+static struct intel_uncore_type ivt_uncore_imc = {
+        .name           = "imc",
+        .num_counters   = 4,
+        .num_boxes      = 8,
+        .perf_ctr_bits  = 48,
+        .fixed_ctr_bits = 48,
+        .fixed_ctr      = SNBEP_MC_CHy_PCI_PMON_FIXED_CTR,
+        .fixed_ctl      = SNBEP_MC_CHy_PCI_PMON_FIXED_CTL,
+        IVT_UNCORE_PCI_COMMON_INIT(),
+};
+
+static struct intel_uncore_type ivt_uncore_qpi = {
+        .name           = "qpi",
+        .num_counters   = 4,
+        .num_boxes      = 3,
+        .perf_ctr_bits  = 48,
+        .perf_ctr       = SNBEP_PCI_PMON_CTR0,
+        .event_ctl      = SNBEP_PCI_PMON_CTL0,
+        .event_mask     = IVT_QPI_PCI_PMON_RAW_EVENT_MASK,
+        .box_ctl        = SNBEP_PCI_PMON_BOX_CTL,
+        .ops            = &ivt_uncore_pci_ops,
+        .format_group   = &ivt_uncore_qpi_format_group,
+};
+
+static struct intel_uncore_type ivt_uncore_r2pcie = {
+        .name           = "r2pcie",
+        .num_counters   = 4,
+        .num_boxes      = 1,
+        .perf_ctr_bits  = 44,
+        .constraints    = snbep_uncore_r2pcie_constraints,
+        IVT_UNCORE_PCI_COMMON_INIT(),
+};
+
+static struct intel_uncore_type ivt_uncore_r3qpi = {
+        .name           = "r3qpi",
+        .num_counters   = 3,
+        .num_boxes      = 2,
+        .perf_ctr_bits  = 44,
+        .constraints    = snbep_uncore_r3qpi_constraints,
+        IVT_UNCORE_PCI_COMMON_INIT(),
+};
+
+enum {
+        IVT_PCI_UNCORE_HA,
+        IVT_PCI_UNCORE_IMC,
+        IVT_PCI_UNCORE_QPI,
+        IVT_PCI_UNCORE_R2PCIE,
+        IVT_PCI_UNCORE_R3QPI,
+};
+
+static struct intel_uncore_type *ivt_pci_uncores[] = {
+        [IVT_PCI_UNCORE_HA]     = &ivt_uncore_ha,
+        [IVT_PCI_UNCORE_IMC]    = &ivt_uncore_imc,
+        [IVT_PCI_UNCORE_QPI]    = &ivt_uncore_qpi,
+        [IVT_PCI_UNCORE_R2PCIE] = &ivt_uncore_r2pcie,
+        [IVT_PCI_UNCORE_R3QPI]  = &ivt_uncore_r3qpi,
+        NULL,
+};
+
+static DEFINE_PCI_DEVICE_TABLE(ivt_uncore_pci_ids) = {
+        { /* Home Agent 0 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xe30),
+                .driver_data = IVT_PCI_UNCORE_HA,
+        },
+        { /* Home Agent 1 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xe38),
+                .driver_data = IVT_PCI_UNCORE_HA,
+        },
+        { /* MC0 Channel 0 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xeb4),
+                .driver_data = IVT_PCI_UNCORE_IMC,
+        },
+        { /* MC0 Channel 1 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xeb5),
+                .driver_data = IVT_PCI_UNCORE_IMC,
+        },
+        { /* MC0 Channel 3 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xeb0),
+                .driver_data = IVT_PCI_UNCORE_IMC,
+        },
+        { /* MC0 Channel 4 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xeb1),
+                .driver_data = IVT_PCI_UNCORE_IMC,
+        },
+        { /* MC1 Channel 0 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xef4),
+                .driver_data = IVT_PCI_UNCORE_IMC,
+        },
+        { /* MC1 Channel 1 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xef5),
+                .driver_data = IVT_PCI_UNCORE_IMC,
+        },
+        { /* MC1 Channel 3 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xef0),
+                .driver_data = IVT_PCI_UNCORE_IMC,
+        },
+        { /* MC1 Channel 4 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xef1),
+                .driver_data = IVT_PCI_UNCORE_IMC,
+        },
+        { /* QPI0 Port 0 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xe32),
+                .driver_data = IVT_PCI_UNCORE_QPI,
+        },
+        { /* QPI0 Port 1 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xe33),
+                .driver_data = IVT_PCI_UNCORE_QPI,
+        },
+        { /* QPI1 Port 2 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xe3a),
+                .driver_data = IVT_PCI_UNCORE_QPI,
+        },
+        { /* R2PCIe */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xe34),
+                .driver_data = IVT_PCI_UNCORE_R2PCIE,
+        },
+        { /* R3QPI0 Link 0 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xe36),
+                .driver_data = IVT_PCI_UNCORE_R3QPI,
+        },
+        { /* R3QPI0 Link 1 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xe37),
+                .driver_data = IVT_PCI_UNCORE_R3QPI,
+        },
+        { /* R3QPI1 Link 2 */
+                PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0xe3e),
+                .driver_data = IVT_PCI_UNCORE_R3QPI,
+        },
+        { /* end: all zeroes */ }
+};
+
+static struct pci_driver ivt_uncore_pci_driver = {
+        .name           = "ivt_uncore",
+        .id_table       = ivt_uncore_pci_ids,
+};
+/* end of IvyTown uncore support */
+
 /* Sandy Bridge uncore support */
 static void snb_uncore_msr_enable_event(struct intel_uncore_box *box, struct perf_event *event)
 {
@@ -808,9 +1491,6 @@ static struct intel_uncore_type *nhm_msr_uncores[] = {
 /* end of Nehalem uncore support */
 
 /* Nehalem-EX uncore support */
-#define __BITS_VALUE(x, i, n)  ((typeof(x))(((x) >> ((i) * (n))) & \
-                                ((1ULL << (n)) - 1)))
-
 DEFINE_UNCORE_FORMAT_ATTR(event5, event, "config:1-5");
 DEFINE_UNCORE_FORMAT_ATTR(counter, counter, "config:6-7");
 DEFINE_UNCORE_FORMAT_ATTR(match, match, "config1:0-63");
@@ -1161,7 +1841,7 @@ static struct extra_reg nhmex_uncore_mbox_extra_regs[] = {
 };
 
 /* Nehalem-EX or Westmere-EX ? */
-bool uncore_nhmex;
+static bool uncore_nhmex;
 
 static bool nhmex_mbox_get_shared_reg(struct intel_uncore_box *box, int idx, u64 config)
 {
@@ -1239,7 +1919,7 @@ static void nhmex_mbox_put_shared_reg(struct intel_uncore_box *box, int idx)
         atomic_sub(1 << (idx * 8), &er->ref);
 }
 
-u64 nhmex_mbox_alter_er(struct perf_event *event, int new_idx, bool modify)
+static u64 nhmex_mbox_alter_er(struct perf_event *event, int new_idx, bool modify)
 {
         struct hw_perf_event *hwc = &event->hw;
         struct hw_perf_event_extra *reg1 = &hwc->extra_reg;
@@ -1554,7 +2234,7 @@ static struct intel_uncore_type nhmex_uncore_mbox = {
         .format_group           = &nhmex_uncore_mbox_format_group,
 };
 
-void nhmex_rbox_alter_er(struct intel_uncore_box *box, struct perf_event *event)
+static void nhmex_rbox_alter_er(struct intel_uncore_box *box, struct perf_event *event)
 {
         struct hw_perf_event *hwc = &event->hw;
         struct hw_perf_event_extra *reg1 = &hwc->extra_reg;
@@ -1724,21 +2404,6 @@ static int nhmex_rbox_hw_config(struct intel_uncore_box *box, struct perf_event
         return 0;
 }
 
-static u64 nhmex_rbox_shared_reg_config(struct intel_uncore_box *box, int idx)
-{
-        struct intel_uncore_extra_reg *er;
-        unsigned long flags;
-        u64 config;
-
-        er = &box->shared_regs[idx];
-
-        raw_spin_lock_irqsave(&er->lock, flags);
-        config = er->config;
-        raw_spin_unlock_irqrestore(&er->lock, flags);
-
-        return config;
-}
-
 static void nhmex_rbox_msr_enable_event(struct intel_uncore_box *box, struct perf_event *event)
 {
         struct hw_perf_event *hwc = &event->hw;
@@ -1759,7 +2424,7 @@ static void nhmex_rbox_msr_enable_event(struct intel_uncore_box *box, struct per
         case 2:
         case 3:
                 wrmsrl(NHMEX_R_MSR_PORTN_QLX_CFG(port),
-                        nhmex_rbox_shared_reg_config(box, 2 + (idx / 6) * 5));
+                        uncore_shared_reg_config(box, 2 + (idx / 6) * 5));
                 break;
         case 4:
                 wrmsrl(NHMEX_R_MSR_PORTN_XBR_SET1_MM_CFG(port),
@@ -2285,7 +2950,7 @@ out:
         return ret;
 }
 
-int uncore_pmu_event_init(struct perf_event *event)
+static int uncore_pmu_event_init(struct perf_event *event)
 {
         struct intel_uncore_pmu *pmu;
         struct intel_uncore_box *box;
@@ -2438,7 +3103,7 @@ static int __init uncore_type_init(struct intel_uncore_type *type)
 
         type->unconstrainted = (struct event_constraint)
                 __EVENT_CONSTRAINT(0, (1ULL << type->num_counters) - 1,
-                                0, type->num_counters, 0);
+                                0, type->num_counters, 0, 0);
 
         for (i = 0; i < type->num_boxes; i++) {
                 pmus[i].func_id = -1;
@@ -2556,6 +3221,8 @@ static void uncore_pci_remove(struct pci_dev *pdev)
         if (WARN_ON_ONCE(phys_id != box->phys_id))
                 return;
 
+        pci_set_drvdata(pdev, NULL);
+
         raw_spin_lock(&uncore_box_lock);
         list_del(&box->list);
         raw_spin_unlock(&uncore_box_lock);
@@ -2574,11 +3241,7 @@ static void uncore_pci_remove(struct pci_dev *pdev)
 static int uncore_pci_probe(struct pci_dev *pdev,
                             const struct pci_device_id *id)
 {
-        struct intel_uncore_type *type;
-
-        type = (struct intel_uncore_type *)id->driver_data;
-
-        return uncore_pci_add(type, pdev);
+        return uncore_pci_add(pci_uncores[id->driver_data], pdev);
 }
 
 static int __init uncore_pci_init(void)
@@ -2587,12 +3250,19 @@ static int __init uncore_pci_init(void)
 
         switch (boot_cpu_data.x86_model) {
         case 45: /* Sandy Bridge-EP */
-                ret = snbep_pci2phy_map_init();
+                ret = snbep_pci2phy_map_init(0x3ce0);
                 if (ret)
                         return ret;
                 pci_uncores = snbep_pci_uncores;
                 uncore_pci_driver = &snbep_uncore_pci_driver;
                 break;
+        case 62: /* IvyTown */
+                ret = snbep_pci2phy_map_init(0x0e1e);
+                if (ret)
+                        return ret;
+                pci_uncores = ivt_pci_uncores;
+                uncore_pci_driver = &ivt_uncore_pci_driver;
+                break;
         default:
                 return 0;
         }
@@ -2622,6 +3292,21 @@ static void __init uncore_pci_exit(void)
         }
 }
 
+/* CPU hot plug/unplug are serialized by cpu_add_remove_lock mutex */
+static LIST_HEAD(boxes_to_free);
+
+static void __cpuinit uncore_kfree_boxes(void)
+{
+        struct intel_uncore_box *box;
+
+        while (!list_empty(&boxes_to_free)) {
+                box = list_entry(boxes_to_free.next,
+                                 struct intel_uncore_box, list);
+                list_del(&box->list);
+                kfree(box);
+        }
+}
+
 static void __cpuinit uncore_cpu_dying(int cpu)
 {
         struct intel_uncore_type *type;
@@ -2636,7 +3321,7 @@ static void __cpuinit uncore_cpu_dying(int cpu)
                         box = *per_cpu_ptr(pmu->box, cpu);
                         *per_cpu_ptr(pmu->box, cpu) = NULL;
                         if (box && atomic_dec_and_test(&box->refcnt))
-                                kfree(box);
+                                list_add(&box->list, &boxes_to_free);
                 }
         }
 }
@@ -2666,8 +3351,11 @@ static int __cpuinit uncore_cpu_starting(int cpu)
                                 if (exist && exist->phys_id == phys_id) {
                                         atomic_inc(&exist->refcnt);
                                         *per_cpu_ptr(pmu->box, cpu) = exist;
-                                        kfree(box);
-                                        box = NULL;
+                                        if (box) {
+                                                list_add(&box->list,
+                                                         &boxes_to_free);
+                                                box = NULL;
+                                        }
                                         break;
                                 }
                         }
@@ -2806,6 +3494,10 @@ static int
         case CPU_DYING:
                 uncore_cpu_dying(cpu);
                 break;
+        case CPU_ONLINE:
+        case CPU_DEAD:
+                uncore_kfree_boxes();
+                break;
         default:
                 break;
         }
@@ -2871,6 +3563,12 @@ static int __init uncore_cpu_init(void)
                         nhmex_uncore_cbox.num_boxes = max_cores;
                 msr_uncores = nhmex_msr_uncores;
                 break;
+        case 62: /* IvyTown */
+                if (ivt_uncore_cbox.num_boxes > max_cores)
+                        ivt_uncore_cbox.num_boxes = max_cores;
+                msr_uncores = ivt_msr_uncores;
+                break;
+
         default:
                 return 0;
         }
diff --git a/arch/x86/kernel/cpu/perf_event_intel_uncore.h b/arch/x86/kernel/cpu/perf_event_intel_uncore.h
index e68a4550e952..f9528917f6e8 100644
--- a/arch/x86/kernel/cpu/perf_event_intel_uncore.h
+++ b/arch/x86/kernel/cpu/perf_event_intel_uncore.h
@@ -76,7 +76,7 @@
 #define SNBEP_PMON_CTL_UMASK_MASK       0x0000ff00
 #define SNBEP_PMON_CTL_RST              (1 << 17)
 #define SNBEP_PMON_CTL_EDGE_DET         (1 << 18)
-#define SNBEP_PMON_CTL_EV_SEL_EXT       (1 << 21)       /* only for QPI */
+#define SNBEP_PMON_CTL_EV_SEL_EXT       (1 << 21)
 #define SNBEP_PMON_CTL_EN               (1 << 22)
 #define SNBEP_PMON_CTL_INVERT           (1 << 23)
 #define SNBEP_PMON_CTL_TRESH_MASK       0xff000000
@@ -148,9 +148,20 @@
 #define SNBEP_C0_MSR_PMON_CTL0                  0xd10
 #define SNBEP_C0_MSR_PMON_BOX_CTL               0xd04
 #define SNBEP_C0_MSR_PMON_BOX_FILTER            0xd14
-#define SNBEP_CB0_MSR_PMON_BOX_FILTER_MASK      0xfffffc1f
 #define SNBEP_CBO_MSR_OFFSET                    0x20
 
+#define SNBEP_CB0_MSR_PMON_BOX_FILTER_TID       0x1f
+#define SNBEP_CB0_MSR_PMON_BOX_FILTER_NID       0x3fc00
+#define SNBEP_CB0_MSR_PMON_BOX_FILTER_STATE     0x7c0000
+#define SNBEP_CB0_MSR_PMON_BOX_FILTER_OPC       0xff800000
+
+#define SNBEP_CBO_EVENT_EXTRA_REG(e, m, i) {    \
+        .event = (e),                           \
+        .msr = SNBEP_C0_MSR_PMON_BOX_FILTER,    \
+        .config_mask = (m),                     \
+        .idx = (i)                              \
+}
+
 /* SNB-EP PCU register */
 #define SNBEP_PCU_MSR_PMON_CTR0                 0xc36
 #define SNBEP_PCU_MSR_PMON_CTL0                 0xc30
@@ -160,6 +171,55 @@
 #define SNBEP_PCU_MSR_CORE_C3_CTR               0x3fc
 #define SNBEP_PCU_MSR_CORE_C6_CTR               0x3fd
 
+/* IVT event control */
+#define IVT_PMON_BOX_CTL_INT            (SNBEP_PMON_BOX_CTL_RST_CTRL | \
+                                         SNBEP_PMON_BOX_CTL_RST_CTRS)
+#define IVT_PMON_RAW_EVENT_MASK         (SNBEP_PMON_CTL_EV_SEL_MASK | \
+                                         SNBEP_PMON_CTL_UMASK_MASK | \
+                                         SNBEP_PMON_CTL_EDGE_DET | \
+                                         SNBEP_PMON_CTL_TRESH_MASK)
+/* IVT Ubox */
+#define IVT_U_MSR_PMON_GLOBAL_CTL               0xc00
+#define IVT_U_PMON_GLOBAL_FRZ_ALL               (1 << 31)
+#define IVT_U_PMON_GLOBAL_UNFRZ_ALL             (1 << 29)
+
+#define IVT_U_MSR_PMON_RAW_EVENT_MASK   \
+                                (SNBEP_PMON_CTL_EV_SEL_MASK | \
+                                 SNBEP_PMON_CTL_UMASK_MASK | \
+                                 SNBEP_PMON_CTL_EDGE_DET | \
+                                 SNBEP_U_MSR_PMON_CTL_TRESH_MASK)
+/* IVT Cbo */
+#define IVT_CBO_MSR_PMON_RAW_EVENT_MASK         (IVT_PMON_RAW_EVENT_MASK | \
+                                                 SNBEP_CBO_PMON_CTL_TID_EN)
+
+#define IVT_CB0_MSR_PMON_BOX_FILTER_TID         (0x1fULL << 0)
+#define IVT_CB0_MSR_PMON_BOX_FILTER_LINK        (0xfULL << 5)
+#define IVT_CB0_MSR_PMON_BOX_FILTER_STATE       (0x3fULL << 17)
+#define IVT_CB0_MSR_PMON_BOX_FILTER_NID         (0xffffULL << 32)
+#define IVT_CB0_MSR_PMON_BOX_FILTER_OPC         (0x1ffULL << 52)
+#define IVT_CB0_MSR_PMON_BOX_FILTER_C6          (0x1ULL << 61)
+#define IVT_CB0_MSR_PMON_BOX_FILTER_NC          (0x1ULL << 62)
+#define IVT_CB0_MSR_PMON_BOX_FILTER_IOSC        (0x1ULL << 63)
+
+/* IVT home agent */
+#define IVT_HA_PCI_PMON_CTL_Q_OCC_RST           (1 << 16)
+#define IVT_HA_PCI_PMON_RAW_EVENT_MASK          \
+                                (IVT_PMON_RAW_EVENT_MASK | \
+                                 IVT_HA_PCI_PMON_CTL_Q_OCC_RST)
+/* IVT PCU */
+#define IVT_PCU_MSR_PMON_RAW_EVENT_MASK \
+                                (SNBEP_PMON_CTL_EV_SEL_MASK | \
+                                 SNBEP_PMON_CTL_EV_SEL_EXT | \
+                                 SNBEP_PCU_MSR_PMON_CTL_OCC_SEL_MASK | \
+                                 SNBEP_PMON_CTL_EDGE_DET | \
+                                 SNBEP_PCU_MSR_PMON_CTL_TRESH_MASK | \
+                                 SNBEP_PCU_MSR_PMON_CTL_OCC_INVERT | \
+                                 SNBEP_PCU_MSR_PMON_CTL_OCC_EDGE_DET)
+/* IVT QPI */
+#define IVT_QPI_PCI_PMON_RAW_EVENT_MASK \
+                                (IVT_PMON_RAW_EVENT_MASK | \
+                                 SNBEP_PMON_CTL_EV_SEL_EXT)
+
 /* NHM-EX event control */
 #define NHMEX_PMON_CTL_EV_SEL_MASK      0x000000ff
 #define NHMEX_PMON_CTL_UMASK_MASK       0x0000ff00
diff --git a/arch/x86/kernel/cpu/perf_event_p4.c b/arch/x86/kernel/cpu/perf_event_p4.c
index 92c7e39a079f..3486e6660357 100644
--- a/arch/x86/kernel/cpu/perf_event_p4.c
+++ b/arch/x86/kernel/cpu/perf_event_p4.c
@@ -895,8 +895,8 @@ static void p4_pmu_disable_pebs(void)
          * So at moment let leave metrics turned on forever -- it's
          * ok for now but need to be revisited!
          *
-         * (void)wrmsrl_safe(MSR_IA32_PEBS_ENABLE, (u64)0);
-         * (void)wrmsrl_safe(MSR_P4_PEBS_MATRIX_VERT, (u64)0);
+         * (void)wrmsrl_safe(MSR_IA32_PEBS_ENABLE, 0);
+         * (void)wrmsrl_safe(MSR_P4_PEBS_MATRIX_VERT, 0);
          */
 }
 
@@ -910,8 +910,7 @@ static inline void p4_pmu_disable_event(struct perf_event *event)
          * asserted again and again
          */
         (void)wrmsrl_safe(hwc->config_base,
-                (u64)(p4_config_unpack_cccr(hwc->config)) &
-                        ~P4_CCCR_ENABLE & ~P4_CCCR_OVF & ~P4_CCCR_RESERVED);
+                p4_config_unpack_cccr(hwc->config) & ~P4_CCCR_ENABLE & ~P4_CCCR_OVF & ~P4_CCCR_RESERVED);
 }
 
 static void p4_pmu_disable_all(void)
@@ -957,7 +956,7 @@ static void p4_pmu_enable_event(struct perf_event *event)
         u64 escr_addr, cccr;
 
         bind = &p4_event_bind_map[idx];
-        escr_addr = (u64)bind->escr_msr[thread];
+        escr_addr = bind->escr_msr[thread];
 
         /*
          * - we dont support cascaded counters yet
diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c
index 7bfe318d3d8a..9895a9a41380 100644
--- a/arch/x86/kernel/kprobes/core.c
+++ b/arch/x86/kernel/kprobes/core.c
@@ -353,7 +353,11 @@ int __kprobes __copy_instruction(u8 *dest, u8 *src)
                  * have given.
                  */
                 newdisp = (u8 *) src + (s64) insn.displacement.value - (u8 *) dest;
-                BUG_ON((s64) (s32) newdisp != newdisp); /* Sanity check.  */
+                if ((s64) (s32) newdisp != newdisp) {
+                        pr_err("Kprobes error: new displacement does not fit into s32 (%llx)\n", newdisp);
+                        pr_err("\tSrc: %p, Dest: %p, old disp: %x\n", src, dest, insn.displacement.value);
+                        return 0;
+                }
                 disp = (u8 *) dest + insn_offset_displacement(&insn);
                 *(s32 *) disp = (s32) newdisp;
         }
diff --git a/arch/x86/kernel/uprobes.c b/arch/x86/kernel/uprobes.c
index 0ba4cfb4f412..2ed845928b5f 100644
--- a/arch/x86/kernel/uprobes.c
+++ b/arch/x86/kernel/uprobes.c
@@ -697,3 +697,32 @@ bool arch_uprobe_skip_sstep(struct arch_uprobe *auprobe, struct pt_regs *regs)
                 send_sig(SIGTRAP, current, 0);
         return ret;
 }
+
+unsigned long
+arch_uretprobe_hijack_return_addr(unsigned long trampoline_vaddr, struct pt_regs *regs)
+{
+        int rasize, ncopied;
+        unsigned long orig_ret_vaddr = 0; /* clear high bits for 32-bit apps */
+
+        rasize = is_ia32_task() ? 4 : 8;
+        ncopied = copy_from_user(&orig_ret_vaddr, (void __user *)regs->sp, rasize);
+        if (unlikely(ncopied))
+                return -1;
+
+        /* check whether address has been already hijacked */
+        if (orig_ret_vaddr == trampoline_vaddr)
+                return orig_ret_vaddr;
+
+        ncopied = copy_to_user((void __user *)regs->sp, &trampoline_vaddr, rasize);
+        if (likely(!ncopied))
+                return orig_ret_vaddr;
+
+        if (ncopied != rasize) {
+                pr_err("uprobe: return address clobbered: pid=%d, %%sp=%#lx, "
+                        "%%ip=%#lx\n", current->pid, regs->sp, regs->ip);
+
+                force_sig_info(SIGSEGV, SEND_SIG_FORCED, current);
+        }
+
+        return -1;
+}
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 10b8f23fab0f..92669cd182a6 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -351,4 +351,10 @@ void ftrace_likely_update(struct ftrace_branch_data *f, int val, int expect);
  */
 #define ACCESS_ONCE(x) (*(volatile typeof(x) *)&(x))
 
+/* Ignore/forbid kprobes attach on very low level functions marked by this attribute: */
+#ifdef CONFIG_KPROBES
+# define __kprobes      __attribute__((__section__(".kprobes.text")))
+#else
+# define __kprobes
+#endif
 #endif /* __LINUX_COMPILER_H */
diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h
index 4b6ef4d33cc2..ca1d27a0d6a6 100644
--- a/include/linux/kprobes.h
+++ b/include/linux/kprobes.h
@@ -29,6 +29,7 @@
  *              <jkenisto@us.ibm.com>  and Prasanna S Panchamukhi
  *              <prasanna@in.ibm.com> added function-return probes.
  */
+#include <linux/compiler.h>     /* for __kprobes */
 #include <linux/linkage.h>
 #include <linux/list.h>
 #include <linux/notifier.h>
@@ -49,16 +50,11 @@
 #define KPROBE_REENTER          0x00000004
 #define KPROBE_HIT_SSDONE       0x00000008
 
-/* Attach to insert probes on any functions which should be ignored*/
-#define __kprobes       __attribute__((__section__(".kprobes.text")))
-
 #else /* CONFIG_KPROBES */
 typedef int kprobe_opcode_t;
 struct arch_specific_insn {
         int dummy;
 };
-#define __kprobes
-
 #endif /* CONFIG_KPROBES */
 
 struct kprobe;
diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h
index 1d795df6f4cf..e0373d26c244 100644
--- a/include/linux/perf_event.h
+++ b/include/linux/perf_event.h
@@ -21,7 +21,6 @@
  */
 
 #ifdef CONFIG_PERF_EVENTS
-# include <linux/cgroup.h>
 # include <asm/perf_event.h>
 # include <asm/local64.h>
 #endif
@@ -128,6 +127,7 @@ struct hw_perf_event {
                         int             event_base_rdpmc;
                         int             idx;
                         int             last_cpu;
+                        int             flags;
 
                         struct hw_perf_event_extra extra_reg;
                         struct hw_perf_event_extra branch_reg;
@@ -299,22 +299,7 @@ struct swevent_hlist {
 #define PERF_ATTACH_GROUP       0x02
 #define PERF_ATTACH_TASK        0x04
 
-#ifdef CONFIG_CGROUP_PERF
-/*
- * perf_cgroup_info keeps track of time_enabled for a cgroup.
- * This is a per-cpu dynamically allocated data structure.
- */
-struct perf_cgroup_info {
-        u64                             time;
-        u64                             timestamp;
-};
-
-struct perf_cgroup {
-        struct                          cgroup_subsys_state css;
-        struct                          perf_cgroup_info *info; /* timing info, one per cpu */
-};
-#endif
-
+struct perf_cgroup;
 struct ring_buffer;
 
 /**
@@ -583,11 +568,13 @@ struct perf_sample_data {
                 u32     reserved;
         }                               cpu_entry;
         u64                             period;
+        union  perf_mem_data_src        data_src;
         struct perf_callchain_entry     *callchain;
         struct perf_raw_record          *raw;
         struct perf_branch_stack        *br_stack;
         struct perf_regs_user           regs_user;
         u64                             stack_user_size;
+        u64                             weight;
 };
 
 static inline void perf_sample_data_init(struct perf_sample_data *data,
@@ -601,6 +588,8 @@ static inline void perf_sample_data_init(struct perf_sample_data *data,
         data->regs_user.abi = PERF_SAMPLE_REGS_ABI_NONE;
         data->regs_user.regs = NULL;
         data->stack_user_size = 0;
+        data->weight = 0;
+        data->data_src.val = 0;
 }
 
 extern void perf_output_sample(struct perf_output_handle *handle,
@@ -831,6 +820,7 @@ do {									\
 struct perf_pmu_events_attr {
         struct device_attribute attr;
         u64 id;
+        const char *event_str;
 };
 
 #define PMU_EVENT_ATTR(_name, _var, _id, _show)                         \
diff --git a/include/linux/uprobes.h b/include/linux/uprobes.h
index 02b83db8e2c5..06f28beed7c2 100644
--- a/include/linux/uprobes.h
+++ b/include/linux/uprobes.h
@@ -38,6 +38,8 @@ struct inode;
 #define UPROBE_HANDLER_REMOVE           1
 #define UPROBE_HANDLER_MASK             1
 
+#define MAX_URETPROBE_DEPTH             64
+
 enum uprobe_filter_ctx {
         UPROBE_FILTER_REGISTER,
         UPROBE_FILTER_UNREGISTER,
@@ -46,6 +48,9 @@ enum uprobe_filter_ctx {
 
 struct uprobe_consumer {
         int (*handler)(struct uprobe_consumer *self, struct pt_regs *regs);
+        int (*ret_handler)(struct uprobe_consumer *self,
+                                unsigned long func,
+                                struct pt_regs *regs);
         bool (*filter)(struct uprobe_consumer *self,
                                 enum uprobe_filter_ctx ctx,
                                 struct mm_struct *mm);
@@ -68,6 +73,8 @@ struct uprobe_task {
         enum uprobe_task_state          state;
         struct arch_uprobe_task         autask;
 
+        struct return_instance          *return_instances;
+        unsigned int                    depth;
         struct uprobe                   *active_uprobe;
 
         unsigned long                   xol_vaddr;
@@ -100,6 +107,7 @@ struct uprobes_state {
 extern int __weak set_swbp(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
 extern int __weak set_orig_insn(struct arch_uprobe *aup, struct mm_struct *mm, unsigned long vaddr);
 extern bool __weak is_swbp_insn(uprobe_opcode_t *insn);
+extern bool __weak is_trap_insn(uprobe_opcode_t *insn);
 extern int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
 extern int uprobe_apply(struct inode *inode, loff_t offset, struct uprobe_consumer *uc, bool);
 extern void uprobe_unregister(struct inode *inode, loff_t offset, struct uprobe_consumer *uc);
diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h
index 9fa9c622a7f4..fb104e51496e 100644
--- a/include/uapi/linux/perf_event.h
+++ b/include/uapi/linux/perf_event.h
@@ -132,8 +132,10 @@ enum perf_event_sample_format {
         PERF_SAMPLE_BRANCH_STACK                = 1U << 11,
         PERF_SAMPLE_REGS_USER                   = 1U << 12,
         PERF_SAMPLE_STACK_USER                  = 1U << 13,
+        PERF_SAMPLE_WEIGHT                      = 1U << 14,
+        PERF_SAMPLE_DATA_SRC                    = 1U << 15,
 
-        PERF_SAMPLE_MAX = 1U << 14,             /* non-ABI */
+        PERF_SAMPLE_MAX = 1U << 16,             /* non-ABI */
 };
 
 /*
@@ -443,6 +445,7 @@ struct perf_event_mmap_page {
 #define PERF_RECORD_MISC_GUEST_KERNEL           (4 << 0)
 #define PERF_RECORD_MISC_GUEST_USER             (5 << 0)
 
+#define PERF_RECORD_MISC_MMAP_DATA              (1 << 13)
 /*
  * Indicates that the content of PERF_SAMPLE_IP points to
  * the actual instruction that triggered the event. See also
@@ -588,6 +591,9 @@ enum perf_event_type {
          *      { u64                   size;
          *        char                  data[size];
          *        u64                   dyn_size; } && PERF_SAMPLE_STACK_USER
+         *
+         *      { u64                   weight;   } && PERF_SAMPLE_WEIGHT
+         *      { u64                   data_src;     } && PERF_SAMPLE_DATA_SRC
          * };
          */
         PERF_RECORD_SAMPLE                      = 9,
@@ -613,4 +619,67 @@ enum perf_callchain_context {
 #define PERF_FLAG_FD_OUTPUT             (1U << 1)
 #define PERF_FLAG_PID_CGROUP            (1U << 2) /* pid=cgroup id, per-cpu mode only */
 
+union perf_mem_data_src {
+        __u64 val;
+        struct {
+                __u64   mem_op:5,       /* type of opcode */
+                        mem_lvl:14,     /* memory hierarchy level */
+                        mem_snoop:5,    /* snoop mode */
+                        mem_lock:2,     /* lock instr */
+                        mem_dtlb:7,     /* tlb access */
+                        mem_rsvd:31;
+        };
+};
+
+/* type of opcode (load/store/prefetch,code) */
+#define PERF_MEM_OP_NA          0x01 /* not available */
+#define PERF_MEM_OP_LOAD        0x02 /* load instruction */
+#define PERF_MEM_OP_STORE       0x04 /* store instruction */
+#define PERF_MEM_OP_PFETCH      0x08 /* prefetch */
+#define PERF_MEM_OP_EXEC        0x10 /* code (execution) */
+#define PERF_MEM_OP_SHIFT       0
+
+/* memory hierarchy (memory level, hit or miss) */
+#define PERF_MEM_LVL_NA         0x01  /* not available */
+#define PERF_MEM_LVL_HIT        0x02  /* hit level */
+#define PERF_MEM_LVL_MISS       0x04  /* miss level  */
+#define PERF_MEM_LVL_L1         0x08  /* L1 */
+#define PERF_MEM_LVL_LFB        0x10  /* Line Fill Buffer */
+#define PERF_MEM_LVL_L2         0x20  /* L2 */
+#define PERF_MEM_LVL_L3         0x40  /* L3 */
+#define PERF_MEM_LVL_LOC_RAM    0x80  /* Local DRAM */
+#define PERF_MEM_LVL_REM_RAM1   0x100 /* Remote DRAM (1 hop) */
+#define PERF_MEM_LVL_REM_RAM2   0x200 /* Remote DRAM (2 hops) */
+#define PERF_MEM_LVL_REM_CCE1   0x400 /* Remote Cache (1 hop) */
+#define PERF_MEM_LVL_REM_CCE2   0x800 /* Remote Cache (2 hops) */
+#define PERF_MEM_LVL_IO         0x1000 /* I/O memory */
+#define PERF_MEM_LVL_UNC        0x2000 /* Uncached memory */
+#define PERF_MEM_LVL_SHIFT      5
+
+/* snoop mode */
+#define PERF_MEM_SNOOP_NA       0x01 /* not available */
+#define PERF_MEM_SNOOP_NONE     0x02 /* no snoop */
+#define PERF_MEM_SNOOP_HIT      0x04 /* snoop hit */
+#define PERF_MEM_SNOOP_MISS     0x08 /* snoop miss */
+#define PERF_MEM_SNOOP_HITM     0x10 /* snoop hit modified */
+#define PERF_MEM_SNOOP_SHIFT    19
+
+/* locked instruction */
+#define PERF_MEM_LOCK_NA        0x01 /* not available */
+#define PERF_MEM_LOCK_LOCKED    0x02 /* locked transaction */
+#define PERF_MEM_LOCK_SHIFT     24
+
+/* TLB access */
+#define PERF_MEM_TLB_NA         0x01 /* not available */
+#define PERF_MEM_TLB_HIT        0x02 /* hit level */
+#define PERF_MEM_TLB_MISS       0x04 /* miss level */
+#define PERF_MEM_TLB_L1         0x08 /* L1 */
+#define PERF_MEM_TLB_L2         0x10 /* L2 */
+#define PERF_MEM_TLB_WK         0x20 /* Hardware Walker*/
+#define PERF_MEM_TLB_OS         0x40 /* OS fault handler */
+#define PERF_MEM_TLB_SHIFT      26
+
+#define PERF_MEM_S(a, s) \
+        (((u64)PERF_MEM_##a##_##s) << PERF_MEM_##a##_SHIFT)
+
 #endif /* _UAPI_LINUX_PERF_EVENT_H */
diff --git a/kernel/events/core.c b/kernel/events/core.c
index dce6e13cf9d7..3820e3cefbae 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -37,6 +37,7 @@
 #include <linux/ftrace_event.h>
 #include <linux/hw_breakpoint.h>
 #include <linux/mm_types.h>
+#include <linux/cgroup.h>
 
 #include "internal.h"
 
@@ -234,6 +235,20 @@ static void perf_ctx_unlock(struct perf_cpu_context *cpuctx,
 #ifdef CONFIG_CGROUP_PERF
 
 /*
+ * perf_cgroup_info keeps track of time_enabled for a cgroup.
+ * This is a per-cpu dynamically allocated data structure.
+ */
+struct perf_cgroup_info {
+        u64                             time;
+        u64                             timestamp;
+};
+
+struct perf_cgroup {
+        struct cgroup_subsys_state      css;
+        struct perf_cgroup_info __percpu *info;
+};
+
+/*
  * Must ensure cgroup is pinned (css_get) before calling
  * this function. In other words, we cannot call this function
  * if there is no cgroup event for the current CPU context.
@@ -976,9 +991,15 @@ static void perf_event__header_size(struct perf_event *event)
         if (sample_type & PERF_SAMPLE_PERIOD)
                 size += sizeof(data->period);
 
+        if (sample_type & PERF_SAMPLE_WEIGHT)
+                size += sizeof(data->weight);
+
         if (sample_type & PERF_SAMPLE_READ)
                 size += event->read_size;
 
+        if (sample_type & PERF_SAMPLE_DATA_SRC)
+                size += sizeof(data->data_src.val);
+
         event->header_size = size;
 }
 
@@ -4193,6 +4214,12 @@ void perf_output_sample(struct perf_output_handle *handle,
                 perf_output_sample_ustack(handle,
                                           data->stack_user_size,
                                           data->regs_user.regs);
+
+        if (sample_type & PERF_SAMPLE_WEIGHT)
+                perf_output_put(handle, data->weight);
+
+        if (sample_type & PERF_SAMPLE_DATA_SRC)
+                perf_output_put(handle, data->data_src.val);
 }
 
 void perf_prepare_sample(struct perf_event_header *header,
@@ -4782,6 +4809,9 @@ got_name:
         mmap_event->file_name = name;
         mmap_event->file_size = size;
 
+        if (!(vma->vm_flags & VM_EXEC))
+                mmap_event->event_id.header.misc |= PERF_RECORD_MISC_MMAP_DATA;
+
         mmap_event->event_id.header.size = sizeof(mmap_event->event_id) + size;
 
         rcu_read_lock();
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index a567c8c7ef31..f3569747d629 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -75,6 +75,15 @@ struct uprobe {
         struct arch_uprobe      arch;
 };
 
+struct return_instance {
+        struct uprobe           *uprobe;
+        unsigned long           func;
+        unsigned long           orig_ret_vaddr; /* original return address */
+        bool                    chained;        /* true, if instance is nested */
+
+        struct return_instance  *next;          /* keep as stack */
+};
+
 /*
  * valid_vma: Verify if the specified vma is an executable vma
  * Relax restrictions while unregistering: vm_flags might have
@@ -173,10 +182,31 @@ bool __weak is_swbp_insn(uprobe_opcode_t *insn)
         return *insn == UPROBE_SWBP_INSN;
 }
 
-static void copy_opcode(struct page *page, unsigned long vaddr, uprobe_opcode_t *opcode)
+/**
+ * is_trap_insn - check if instruction is breakpoint instruction.
+ * @insn: instruction to be checked.
+ * Default implementation of is_trap_insn
+ * Returns true if @insn is a breakpoint instruction.
+ *
+ * This function is needed for the case where an architecture has multiple
+ * trap instructions (like powerpc).
+ */
+bool __weak is_trap_insn(uprobe_opcode_t *insn)
+{
+        return is_swbp_insn(insn);
+}
+
+static void copy_from_page(struct page *page, unsigned long vaddr, void *dst, int len)
 {
         void *kaddr = kmap_atomic(page);
-        memcpy(opcode, kaddr + (vaddr & ~PAGE_MASK), UPROBE_SWBP_INSN_SIZE);
+        memcpy(dst, kaddr + (vaddr & ~PAGE_MASK), len);
+        kunmap_atomic(kaddr);
+}
+
+static void copy_to_page(struct page *page, unsigned long vaddr, const void *src, int len)
+{
+        void *kaddr = kmap_atomic(page);
+        memcpy(kaddr + (vaddr & ~PAGE_MASK), src, len);
         kunmap_atomic(kaddr);
 }
 
@@ -185,7 +215,16 @@ static int verify_opcode(struct page *page, unsigned long vaddr, uprobe_opcode_t
         uprobe_opcode_t old_opcode;
         bool is_swbp;
 
-        copy_opcode(page, vaddr, &old_opcode);
+        /*
+         * Note: We only check if the old_opcode is UPROBE_SWBP_INSN here.
+         * We do not check if it is any other 'trap variant' which could
+         * be conditional trap instruction such as the one powerpc supports.
+         *
+         * The logic is that we do not care if the underlying instruction
+         * is a trap variant; uprobes always wins over any other (gdb)
+         * breakpoint.
+         */
+        copy_from_page(page, vaddr, &old_opcode, UPROBE_SWBP_INSN_SIZE);
         is_swbp = is_swbp_insn(&old_opcode);
 
         if (is_swbp_insn(new_opcode)) {
@@ -204,7 +243,7 @@ static int verify_opcode(struct page *page, unsigned long vaddr, uprobe_opcode_t
  * Expect the breakpoint instruction to be the smallest size instruction for
  * the architecture. If an arch has variable length instruction and the
  * breakpoint instruction is not of the smallest length instruction
- * supported by that architecture then we need to modify is_swbp_at_addr and
+ * supported by that architecture then we need to modify is_trap_at_addr and
  * write_opcode accordingly. This would never be a problem for archs that
  * have fixed length instructions.
  */
@@ -225,7 +264,6 @@ static int write_opcode(struct mm_struct *mm, unsigned long vaddr,
                         uprobe_opcode_t opcode)
 {
         struct page *old_page, *new_page;
-        void *vaddr_old, *vaddr_new;
         struct vm_area_struct *vma;
         int ret;
 
@@ -246,15 +284,8 @@ retry:
 
         __SetPageUptodate(new_page);
 
-        /* copy the page now that we've got it stable */
-        vaddr_old = kmap_atomic(old_page);
-        vaddr_new = kmap_atomic(new_page);
-
-        memcpy(vaddr_new, vaddr_old, PAGE_SIZE);
-        memcpy(vaddr_new + (vaddr & ~PAGE_MASK), &opcode, UPROBE_SWBP_INSN_SIZE);
-
-        kunmap_atomic(vaddr_new);
-        kunmap_atomic(vaddr_old);
+        copy_highpage(new_page, old_page);
+        copy_to_page(new_page, vaddr, &opcode, UPROBE_SWBP_INSN_SIZE);
 
         ret = anon_vma_prepare(vma);
         if (ret)
@@ -477,30 +508,18 @@ __copy_insn(struct address_space *mapping, struct file *filp, char *insn,
                         unsigned long nbytes, loff_t offset)
 {
         struct page *page;
-        void *vaddr;
-        unsigned long off;
-        pgoff_t idx;
-
-        if (!filp)
-                return -EINVAL;
 
         if (!mapping->a_ops->readpage)
                 return -EIO;
-
-        idx = offset >> PAGE_CACHE_SHIFT;
-        off = offset & ~PAGE_MASK;
-
         /*
          * Ensure that the page that has the original instruction is
          * populated and in page-cache.
          */
-        page = read_mapping_page(mapping, idx, filp);
+        page = read_mapping_page(mapping, offset >> PAGE_CACHE_SHIFT, filp);
         if (IS_ERR(page))
                 return PTR_ERR(page);
 
-        vaddr = kmap_atomic(page);
-        memcpy(insn, vaddr + off, nbytes);
-        kunmap_atomic(vaddr);
+        copy_from_page(page, offset, insn, nbytes);
         page_cache_release(page);
 
         return 0;
@@ -550,7 +569,7 @@ static int prepare_uprobe(struct uprobe *uprobe, struct file *file,
                 goto out;
 
         ret = -ENOTSUPP;
-        if (is_swbp_insn((uprobe_opcode_t *)uprobe->arch.insn))
+        if (is_trap_insn((uprobe_opcode_t *)uprobe->arch.insn))
                 goto out;
 
         ret = arch_uprobe_analyze_insn(&uprobe->arch, mm, vaddr);
@@ -758,7 +777,7 @@ register_for_each_vma(struct uprobe *uprobe, struct uprobe_consumer *new)
                 down_write(&mm->mmap_sem);
                 vma = find_vma(mm, info->vaddr);
                 if (!vma || !valid_vma(vma, is_register) ||
-                    vma->vm_file->f_mapping->host != uprobe->inode)
+                    file_inode(vma->vm_file) != uprobe->inode)
                         goto unlock;
 
                 if (vma->vm_start > info->vaddr ||
@@ -828,6 +847,10 @@ int uprobe_register(struct inode *inode, loff_t offset, struct uprobe_consumer *
         struct uprobe *uprobe;
         int ret;
 
+        /* Uprobe must have at least one set consumer */
+        if (!uc->handler && !uc->ret_handler)
+                return -EINVAL;
+
         /* Racy, just to catch the obvious mistakes */
         if (offset > i_size_read(inode))
                 return -EINVAL;
@@ -917,7 +940,7 @@ static int unapply_uprobe(struct uprobe *uprobe, struct mm_struct *mm)
                 loff_t offset;
 
                 if (!valid_vma(vma, false) ||
-                    vma->vm_file->f_mapping->host != uprobe->inode)
+                    file_inode(vma->vm_file) != uprobe->inode)
                         continue;
 
                 offset = (loff_t)vma->vm_pgoff << PAGE_SHIFT;
@@ -1010,7 +1033,7 @@ int uprobe_mmap(struct vm_area_struct *vma)
         if (no_uprobe_events() || !valid_vma(vma, true))
                 return 0;
 
-        inode = vma->vm_file->f_mapping->host;
+        inode = file_inode(vma->vm_file);
         if (!inode)
                 return 0;
 
@@ -1041,7 +1064,7 @@ vma_has_uprobes(struct vm_area_struct *vma, unsigned long start, unsigned long e
         struct inode *inode;
         struct rb_node *n;
 
-        inode = vma->vm_file->f_mapping->host;
+        inode = file_inode(vma->vm_file);
 
         min = vaddr_to_offset(vma, start);
         max = min + (end - start) - 1;
@@ -1114,6 +1137,7 @@ static struct xol_area *get_xol_area(void)
 {
         struct mm_struct *mm = current->mm;
         struct xol_area *area;
+        uprobe_opcode_t insn = UPROBE_SWBP_INSN;
 
         area = mm->uprobes_state.xol_area;
         if (area)
@@ -1131,7 +1155,12 @@ static struct xol_area *get_xol_area(void)
         if (!area->page)
                 goto free_bitmap;
 
+        /* allocate first slot of task's xol_area for the return probes */
+        set_bit(0, area->bitmap);
+        copy_to_page(area->page, 0, &insn, UPROBE_SWBP_INSN_SIZE);
+        atomic_set(&area->slot_count, 1);
         init_waitqueue_head(&area->wq);
+
         if (!xol_add_vma(area))
                 return area;
 
@@ -1216,9 +1245,7 @@ static unsigned long xol_take_insn_slot(struct xol_area *area)
 static unsigned long xol_get_insn_slot(struct uprobe *uprobe)
 {
         struct xol_area *area;
-        unsigned long offset;
         unsigned long xol_vaddr;
-        void *vaddr;
 
         area = get_xol_area();
         if (!area)
@@ -1229,10 +1256,7 @@ static unsigned long xol_get_insn_slot(struct uprobe *uprobe)
                 return 0;
 
         /* Initialize the slot */
-        offset = xol_vaddr & ~PAGE_MASK;
-        vaddr = kmap_atomic(area->page);
-        memcpy(vaddr + offset, uprobe->arch.insn, MAX_UINSN_BYTES);
-        kunmap_atomic(vaddr);
+        copy_to_page(area->page, xol_vaddr, uprobe->arch.insn, MAX_UINSN_BYTES);
         /*
          * We probably need flush_icache_user_range() but it needs vma.
          * This should work on supported architectures too.
@@ -1298,6 +1322,7 @@ unsigned long __weak uprobe_get_swbp_addr(struct pt_regs *regs)
 void uprobe_free_utask(struct task_struct *t)
 {
         struct uprobe_task *utask = t->utask;
+        struct return_instance *ri, *tmp;
 
         if (!utask)
                 return;
@@ -1305,6 +1330,15 @@ void uprobe_free_utask(struct task_struct *t)
         if (utask->active_uprobe)
                 put_uprobe(utask->active_uprobe);
 
+        ri = utask->return_instances;
+        while (ri) {
+                tmp = ri;
+                ri = ri->next;
+
+                put_uprobe(tmp->uprobe);
+                kfree(tmp);
+        }
+
         xol_free_insn_slot(t);
         kfree(utask);
         t->utask = NULL;
@@ -1333,6 +1367,93 @@ static struct uprobe_task *get_utask(void)
         return current->utask;
 }
 
+/*
+ * Current area->vaddr notion assume the trampoline address is always
+ * equal area->vaddr.
+ *
+ * Returns -1 in case the xol_area is not allocated.
+ */
+static unsigned long get_trampoline_vaddr(void)
+{
+        struct xol_area *area;
+        unsigned long trampoline_vaddr = -1;
+
+        area = current->mm->uprobes_state.xol_area;
+        smp_read_barrier_depends();
+        if (area)
+                trampoline_vaddr = area->vaddr;
+
+        return trampoline_vaddr;
+}
+
+static void prepare_uretprobe(struct uprobe *uprobe, struct pt_regs *regs)
+{
+        struct return_instance *ri;
+        struct uprobe_task *utask;
+        unsigned long orig_ret_vaddr, trampoline_vaddr;
+        bool chained = false;
+
+        if (!get_xol_area())
+                return;
+
+        utask = get_utask();
+        if (!utask)
+                return;
+
+        if (utask->depth >= MAX_URETPROBE_DEPTH) {
+                printk_ratelimited(KERN_INFO "uprobe: omit uretprobe due to"
+                                " nestedness limit pid/tgid=%d/%d\n",
+                                current->pid, current->tgid);
+                return;
+        }
+
+        ri = kzalloc(sizeof(struct return_instance), GFP_KERNEL);
+        if (!ri)
+                goto fail;
+
+        trampoline_vaddr = get_trampoline_vaddr();
+        orig_ret_vaddr = arch_uretprobe_hijack_return_addr(trampoline_vaddr, regs);
+        if (orig_ret_vaddr == -1)
+                goto fail;
+
+        /*
+         * We don't want to keep trampoline address in stack, rather keep the
+         * original return address of first caller thru all the consequent
+         * instances. This also makes breakpoint unwrapping easier.
+         */
+        if (orig_ret_vaddr == trampoline_vaddr) {
+                if (!utask->return_instances) {
+                        /*
+                         * This situation is not possible. Likely we have an
+                         * attack from user-space.
+                         */
+                        pr_warn("uprobe: unable to set uretprobe pid/tgid=%d/%d\n",
+                                                current->pid, current->tgid);
+                        goto fail;
+                }
+
+                chained = true;
+                orig_ret_vaddr = utask->return_instances->orig_ret_vaddr;
+        }
+
+        atomic_inc(&uprobe->ref);
+        ri->uprobe = uprobe;
+        ri->func = instruction_pointer(regs);
+        ri->orig_ret_vaddr = orig_ret_vaddr;
+        ri->chained = chained;
+
+        utask->depth++;
+
+        /* add instance to the stack */
+        ri->next = utask->return_instances;
+        utask->return_instances = ri;
+
+        return;
+
+ fail:
+        kfree(ri);
+}
+
 /* Prepare to single-step probed instruction out of line. */
 static int
 pre_ssout(struct uprobe *uprobe, struct pt_regs *regs, unsigned long bp_vaddr)
@@ -1431,7 +1552,7 @@ static void mmf_recalc_uprobes(struct mm_struct *mm)
         clear_bit(MMF_HAS_UPROBES, &mm->flags);
 }
 
-static int is_swbp_at_addr(struct mm_struct *mm, unsigned long vaddr)
+static int is_trap_at_addr(struct mm_struct *mm, unsigned long vaddr)
 {
         struct page *page;
         uprobe_opcode_t opcode;
@@ -1449,10 +1570,11 @@ static int is_swbp_at_addr(struct mm_struct *mm, unsigned long vaddr)
         if (result < 0)
                 return result;
 
-        copy_opcode(page, vaddr, &opcode);
+        copy_from_page(page, vaddr, &opcode, UPROBE_SWBP_INSN_SIZE);
         put_page(page);
  out:
-        return is_swbp_insn(&opcode);
+        /* This needs to return true for any variant of the trap insn */
+        return is_trap_insn(&opcode);
 }
 
 static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp)
@@ -1465,14 +1587,14 @@ static struct uprobe *find_active_uprobe(unsigned long bp_vaddr, int *is_swbp)
         vma = find_vma(mm, bp_vaddr);
         if (vma && vma->vm_start <= bp_vaddr) {
                 if (valid_vma(vma, false)) {
-                        struct inode *inode = vma->vm_file->f_mapping->host;
+                        struct inode *inode = file_inode(vma->vm_file);
                         loff_t offset = vaddr_to_offset(vma, bp_vaddr);
 
                         uprobe = find_uprobe(inode, offset);
                 }
 
                 if (!uprobe)
-                        *is_swbp = is_swbp_at_addr(mm, bp_vaddr);
+                        *is_swbp = is_trap_at_addr(mm, bp_vaddr);
         } else {
                 *is_swbp = -EFAULT;
         }
@@ -1488,16 +1610,27 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs)
 {
         struct uprobe_consumer *uc;
         int remove = UPROBE_HANDLER_REMOVE;
+        bool need_prep = false; /* prepare return uprobe, when needed */
 
         down_read(&uprobe->register_rwsem);
         for (uc = uprobe->consumers; uc; uc = uc->next) {
-                int rc = uc->handler(uc, regs);
+                int rc = 0;
+
+                if (uc->handler) {
+                        rc = uc->handler(uc, regs);
+                        WARN(rc & ~UPROBE_HANDLER_MASK,
+                                "bad rc=0x%x from %pf()\n", rc, uc->handler);
+                }
+
+                if (uc->ret_handler)
+                        need_prep = true;
 
-                WARN(rc & ~UPROBE_HANDLER_MASK,
-                        "bad rc=0x%x from %pf()\n", rc, uc->handler);
                 remove &= rc;
         }
 
+        if (need_prep && !remove)
+                prepare_uretprobe(uprobe, regs); /* put bp at return */
+
         if (remove && uprobe->consumers) {
                 WARN_ON(!uprobe_is_active(uprobe));
                 unapply_uprobe(uprobe, current->mm);
@@ -1505,6 +1638,64 @@ static void handler_chain(struct uprobe *uprobe, struct pt_regs *regs)
         up_read(&uprobe->register_rwsem);
 }
 
+static void
+handle_uretprobe_chain(struct return_instance *ri, struct pt_regs *regs)
+{
+        struct uprobe *uprobe = ri->uprobe;
+        struct uprobe_consumer *uc;
+
+        down_read(&uprobe->register_rwsem);
+        for (uc = uprobe->consumers; uc; uc = uc->next) {
+                if (uc->ret_handler)
+                        uc->ret_handler(uc, ri->func, regs);
+        }
+        up_read(&uprobe->register_rwsem);
+}
+
+static bool handle_trampoline(struct pt_regs *regs)
+{
+        struct uprobe_task *utask;
+        struct return_instance *ri, *tmp;
+        bool chained;
+
+        utask = current->utask;
+        if (!utask)
+                return false;
+
+        ri = utask->return_instances;
+        if (!ri)
+                return false;
+
+        /*
+         * TODO: we should throw out return_instance's invalidated by
+         * longjmp(), currently we assume that the probed function always
+         * returns.
+         */
+        instruction_pointer_set(regs, ri->orig_ret_vaddr);
+
+        for (;;) {
+                handle_uretprobe_chain(ri, regs);
+
+                chained = ri->chained;
+                put_uprobe(ri->uprobe);
+
+                tmp = ri;
+                ri = ri->next;
+                kfree(tmp);
+
+                if (!chained)
+                        break;
+
+                utask->depth--;
+
+                BUG_ON(!ri);
+        }
+
+        utask->return_instances = ri;
+
+        return true;
+}
+
 /*
  * Run handler and ask thread to singlestep.
  * Ensure all non-fatal signals cannot interrupt thread while it singlesteps.
@@ -1516,8 +1707,15 @@ static void handle_swbp(struct pt_regs *regs)
         int uninitialized_var(is_swbp);
 
         bp_vaddr = uprobe_get_swbp_addr(regs);
-        uprobe = find_active_uprobe(bp_vaddr, &is_swbp);
+        if (bp_vaddr == get_trampoline_vaddr()) {
+                if (handle_trampoline(regs))
+                        return;
+
+                pr_warn("uprobe: unable to handle uretprobe pid/tgid=%d/%d\n",
+                                                current->pid, current->tgid);
+        }
 
+        uprobe = find_active_uprobe(bp_vaddr, &is_swbp);
         if (!uprobe) {
                 if (is_swbp > 0) {
                         /* No matching uprobe; signal SIGTRAP. */
@@ -1616,7 +1814,11 @@ void uprobe_notify_resume(struct pt_regs *regs)
  */
 int uprobe_pre_sstep_notifier(struct pt_regs *regs)
 {
-        if (!current->mm || !test_bit(MMF_HAS_UPROBES, &current->mm->flags))
+        if (!current->mm)
+                return 0;
+
+        if (!test_bit(MMF_HAS_UPROBES, &current->mm->flags) &&
+            (!current->utask || !current->utask->return_instances))
                 return 0;
 
         set_thread_flag(TIF_UPROBE);
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h
index 9e014582e763..711ca7d3e7f1 100644
--- a/kernel/trace/trace.h
+++ b/kernel/trace/trace.h
@@ -109,11 +109,6 @@ struct kretprobe_trace_entry_head {
         unsigned long           ret_ip;
 };
 
-struct uprobe_trace_entry_head {
-        struct trace_entry      ent;
-        unsigned long           ip;
-};
-
 /*
  * trace_flag_type is an enumeration that holds different
  * states when a trace occurs. These are:
diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
index 8dad2a92dee9..32494fb0ee64 100644
--- a/kernel/trace/trace_uprobe.c
+++ b/kernel/trace/trace_uprobe.c
@@ -28,6 +28,18 @@
 
 #define UPROBE_EVENT_SYSTEM     "uprobes"
 
+struct uprobe_trace_entry_head {
+        struct trace_entry      ent;
+        unsigned long           vaddr[];
+};
+
+#define SIZEOF_TRACE_ENTRY(is_return)                   \
+        (sizeof(struct uprobe_trace_entry_head) +       \
+         sizeof(unsigned long) * (is_return ? 2 : 1))
+
+#define DATAOF_TRACE_ENTRY(entry, is_return)            \
+        ((void*)(entry) + SIZEOF_TRACE_ENTRY(is_return))
+
 struct trace_uprobe_filter {
         rwlock_t                rwlock;
         int                     nr_systemwide;
@@ -64,6 +76,8 @@ static DEFINE_MUTEX(uprobe_lock);
 static LIST_HEAD(uprobe_list);
 
 static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs);
+static int uretprobe_dispatcher(struct uprobe_consumer *con,
+                                unsigned long func, struct pt_regs *regs);
 
 static inline void init_trace_uprobe_filter(struct trace_uprobe_filter *filter)
 {
@@ -77,11 +91,16 @@ static inline bool uprobe_filter_is_empty(struct trace_uprobe_filter *filter)
         return !filter->nr_systemwide && list_empty(&filter->perf_events);
 }
 
+static inline bool is_ret_probe(struct trace_uprobe *tu)
+{
+        return tu->consumer.ret_handler != NULL;
+}
+
 /*
  * Allocate new trace_uprobe and initialize it (including uprobes).
  */
 static struct trace_uprobe *
-alloc_trace_uprobe(const char *group, const char *event, int nargs)
+alloc_trace_uprobe(const char *group, const char *event, int nargs, bool is_ret)
 {
         struct trace_uprobe *tu;
 
@@ -106,6 +125,8 @@ alloc_trace_uprobe(const char *group, const char *event, int nargs)
 
         INIT_LIST_HEAD(&tu->list);
         tu->consumer.handler = uprobe_dispatcher;
+        if (is_ret)
+                tu->consumer.ret_handler = uretprobe_dispatcher;
         init_trace_uprobe_filter(&tu->filter);
         return tu;
 
@@ -180,7 +201,7 @@ end:
 
 /*
  * Argument syntax:
- *  - Add uprobe: p[:[GRP/]EVENT] PATH:SYMBOL[+offs] [FETCHARGS]
+ *  - Add uprobe: p|r[:[GRP/]EVENT] PATH:SYMBOL [FETCHARGS]
  *
  *  - Remove uprobe: -:[GRP/]EVENT
  */
@@ -192,20 +213,23 @@ static int create_trace_uprobe(int argc, char **argv)
         char buf[MAX_EVENT_NAME_LEN];
         struct path path;
         unsigned long offset;
-        bool is_delete;
+        bool is_delete, is_return;
         int i, ret;
 
         inode = NULL;
         ret = 0;
         is_delete = false;
+        is_return = false;
         event = NULL;
         group = NULL;
 
         /* argc must be >= 1 */
         if (argv[0][0] == '-')
                 is_delete = true;
+        else if (argv[0][0] == 'r')
+                is_return = true;
         else if (argv[0][0] != 'p') {
-                pr_info("Probe definition must be started with 'p' or '-'.\n");
+                pr_info("Probe definition must be started with 'p', 'r' or '-'.\n");
                 return -EINVAL;
         }
 
@@ -303,7 +327,7 @@ static int create_trace_uprobe(int argc, char **argv)
                 kfree(tail);
         }
 
-        tu = alloc_trace_uprobe(group, event, argc);
+        tu = alloc_trace_uprobe(group, event, argc, is_return);
         if (IS_ERR(tu)) {
                 pr_info("Failed to allocate trace_uprobe.(%d)\n", (int)PTR_ERR(tu));
                 ret = PTR_ERR(tu);
@@ -414,9 +438,10 @@ static void probes_seq_stop(struct seq_file *m, void *v)
 static int probes_seq_show(struct seq_file *m, void *v)
 {
         struct trace_uprobe *tu = v;
+        char c = is_ret_probe(tu) ? 'r' : 'p';
         int i;
 
-        seq_printf(m, "p:%s/%s", tu->call.class->system, tu->call.name);
+        seq_printf(m, "%c:%s/%s", c, tu->call.class->system, tu->call.name);
         seq_printf(m, " %s:0x%p", tu->filename, (void *)tu->offset);
 
         for (i = 0; i < tu->nr_args; i++)
@@ -485,65 +510,81 @@ static const struct file_operations uprobe_profile_ops = {
         .release        = seq_release,
 };
 
-/* uprobe handler */
-static int uprobe_trace_func(struct trace_uprobe *tu, struct pt_regs *regs)
+static void uprobe_trace_print(struct trace_uprobe *tu,
+                                unsigned long func, struct pt_regs *regs)
 {
         struct uprobe_trace_entry_head *entry;
         struct ring_buffer_event *event;
         struct ring_buffer *buffer;
-        u8 *data;
-        int size, i, pc;
-        unsigned long irq_flags;
+        void *data;
+        int size, i;
         struct ftrace_event_call *call = &tu->call;
 
-        local_save_flags(irq_flags);
-        pc = preempt_count();
-
-        size = sizeof(*entry) + tu->size;
-
+        size = SIZEOF_TRACE_ENTRY(is_ret_probe(tu));
         event = trace_current_buffer_lock_reserve(&buffer, call->event.type,
-                                                  size, irq_flags, pc);
+                                                  size + tu->size, 0, 0);
         if (!event)
-                return 0;
+                return;
 
         entry = ring_buffer_event_data(event);
-        entry->ip = instruction_pointer(task_pt_regs(current));
-        data = (u8 *)&entry[1];
+        if (is_ret_probe(tu)) {
+                entry->vaddr[0] = func;
+                entry->vaddr[1] = instruction_pointer(regs);
+                data = DATAOF_TRACE_ENTRY(entry, true);
+        } else {
+                entry->vaddr[0] = instruction_pointer(regs);
+                data = DATAOF_TRACE_ENTRY(entry, false);
+        }
+
         for (i = 0; i < tu->nr_args; i++)
                 call_fetch(&tu->args[i].fetch, regs, data + tu->args[i].offset);
 
         if (!filter_current_check_discard(buffer, call, entry, event))
-                trace_buffer_unlock_commit(buffer, event, irq_flags, pc);
+                trace_buffer_unlock_commit(buffer, event, 0, 0);
+}
 
+/* uprobe handler */
+static int uprobe_trace_func(struct trace_uprobe *tu, struct pt_regs *regs)
+{
+        if (!is_ret_probe(tu))
+                uprobe_trace_print(tu, 0, regs);
         return 0;
 }
 
+static void uretprobe_trace_func(struct trace_uprobe *tu, unsigned long func,
+                                struct pt_regs *regs)
+{
+        uprobe_trace_print(tu, func, regs);
+}
+
 /* Event entry printers */
 static enum print_line_t
 print_uprobe_event(struct trace_iterator *iter, int flags, struct trace_event *event)
 {
-        struct uprobe_trace_entry_head *field;
+        struct uprobe_trace_entry_head *entry;
         struct trace_seq *s = &iter->seq;
         struct trace_uprobe *tu;
         u8 *data;
         int i;
 
-        field = (struct uprobe_trace_entry_head *)iter->ent;
+        entry = (struct uprobe_trace_entry_head *)iter->ent;
         tu = container_of(event, struct trace_uprobe, call.event);
 
-        if (!trace_seq_printf(s, "%s: (", tu->call.name))
-                goto partial;
-
-        if (!seq_print_ip_sym(s, field->ip, flags | TRACE_ITER_SYM_OFFSET))
-                goto partial;
-
-        if (!trace_seq_puts(s, ")"))
-                goto partial;
+        if (is_ret_probe(tu)) {
+                if (!trace_seq_printf(s, "%s: (0x%lx <- 0x%lx)", tu->call.name,
+                                        entry->vaddr[1], entry->vaddr[0]))
+                        goto partial;
+                data = DATAOF_TRACE_ENTRY(entry, true);
+        } else {
+                if (!trace_seq_printf(s, "%s: (0x%lx)", tu->call.name,
+                                        entry->vaddr[0]))
+                        goto partial;
+                data = DATAOF_TRACE_ENTRY(entry, false);
+        }
 
-        data = (u8 *)&field[1];
         for (i = 0; i < tu->nr_args; i++) {
                 if (!tu->args[i].type->print(s, tu->args[i].name,
-                                             data + tu->args[i].offset, field))
+                                             data + tu->args[i].offset, entry))
                         goto partial;
         }
 
@@ -595,16 +636,23 @@ static void probe_event_disable(struct trace_uprobe *tu, int flag)
 
 static int uprobe_event_define_fields(struct ftrace_event_call *event_call)
 {
-        int ret, i;
+        int ret, i, size;
         struct uprobe_trace_entry_head field;
-        struct trace_uprobe *tu = (struct trace_uprobe *)event_call->data;
+        struct trace_uprobe *tu = event_call->data;
 
-        DEFINE_FIELD(unsigned long, ip, FIELD_STRING_IP, 0);
+        if (is_ret_probe(tu)) {
+                DEFINE_FIELD(unsigned long, vaddr[0], FIELD_STRING_FUNC, 0);
+                DEFINE_FIELD(unsigned long, vaddr[1], FIELD_STRING_RETIP, 0);
+                size = SIZEOF_TRACE_ENTRY(true);
+        } else {
+                DEFINE_FIELD(unsigned long, vaddr[0], FIELD_STRING_IP, 0);
+                size = SIZEOF_TRACE_ENTRY(false);
+        }
         /* Set argument names as fields */
         for (i = 0; i < tu->nr_args; i++) {
                 ret = trace_define_field(event_call, tu->args[i].type->fmttype,
                                          tu->args[i].name,
-                                         sizeof(field) + tu->args[i].offset,
+                                         size + tu->args[i].offset,
                                          tu->args[i].type->size,
                                          tu->args[i].type->is_signed,
                                          FILTER_OTHER);
@@ -622,8 +670,13 @@ static int __set_print_fmt(struct trace_uprobe *tu, char *buf, int len)
         int i;
         int pos = 0;
 
-        fmt = "(%lx)";
-        arg = "REC->" FIELD_STRING_IP;
+        if (is_ret_probe(tu)) {
+                fmt = "(%lx <- %lx)";
+                arg = "REC->" FIELD_STRING_FUNC ", REC->" FIELD_STRING_RETIP;
+        } else {
+                fmt = "(%lx)";
+                arg = "REC->" FIELD_STRING_IP;
+        }
 
         /* When len=0, we just calculate the needed length */
 
@@ -752,49 +805,68 @@ static bool uprobe_perf_filter(struct uprobe_consumer *uc,
         return ret;
 }
 
-/* uprobe profile handler */
-static int uprobe_perf_func(struct trace_uprobe *tu, struct pt_regs *regs)
+static void uprobe_perf_print(struct trace_uprobe *tu,
+                                unsigned long func, struct pt_regs *regs)
 {
         struct ftrace_event_call *call = &tu->call;
         struct uprobe_trace_entry_head *entry;
         struct hlist_head *head;
-        u8 *data;
-        int size, __size, i;
-        int rctx;
+        void *data;
+        int size, rctx, i;
 
-        if (!uprobe_perf_filter(&tu->consumer, 0, current->mm))
-                return UPROBE_HANDLER_REMOVE;
-
-        __size = sizeof(*entry) + tu->size;
-        size = ALIGN(__size + sizeof(u32), sizeof(u64));
-        size -= sizeof(u32);
+        size = SIZEOF_TRACE_ENTRY(is_ret_probe(tu));
+        size = ALIGN(size + tu->size + sizeof(u32), sizeof(u64)) - sizeof(u32);
         if (WARN_ONCE(size > PERF_MAX_TRACE_SIZE, "profile buffer not large enough"))
-                return 0;
+                return;
 
         preempt_disable();
+        head = this_cpu_ptr(call->perf_events);
+        if (hlist_empty(head))
+                goto out;
 
         entry = perf_trace_buf_prepare(size, call->event.type, regs, &rctx);
         if (!entry)
                 goto out;
 
-        entry->ip = instruction_pointer(task_pt_regs(current));
-        data = (u8 *)&entry[1];
+        if (is_ret_probe(tu)) {
+                entry->vaddr[0] = func;
+                entry->vaddr[1] = instruction_pointer(regs);
+                data = DATAOF_TRACE_ENTRY(entry, true);
+        } else {
+                entry->vaddr[0] = instruction_pointer(regs);
+                data = DATAOF_TRACE_ENTRY(entry, false);
+        }
+
         for (i = 0; i < tu->nr_args; i++)
                 call_fetch(&tu->args[i].fetch, regs, data + tu->args[i].offset);
 
-        head = this_cpu_ptr(call->perf_events);
-        perf_trace_buf_submit(entry, size, rctx, entry->ip, 1, regs, head, NULL);
-
+        perf_trace_buf_submit(entry, size, rctx, 0, 1, regs, head, NULL);
  out:
         preempt_enable();
+}
+
+/* uprobe profile handler */
+static int uprobe_perf_func(struct trace_uprobe *tu, struct pt_regs *regs)
+{
+        if (!uprobe_perf_filter(&tu->consumer, 0, current->mm))
+                return UPROBE_HANDLER_REMOVE;
+
+        if (!is_ret_probe(tu))
+                uprobe_perf_print(tu, 0, regs);
         return 0;
 }
+
+static void uretprobe_perf_func(struct trace_uprobe *tu, unsigned long func,
+                                struct pt_regs *regs)
+{
+        uprobe_perf_print(tu, func, regs);
+}
 #endif  /* CONFIG_PERF_EVENTS */
 
 static
 int trace_uprobe_register(struct ftrace_event_call *event, enum trace_reg type, void *data)
 {
-        struct trace_uprobe *tu = (struct trace_uprobe *)event->data;
+        struct trace_uprobe *tu = event->data;
 
         switch (type) {
         case TRACE_REG_REGISTER:
@@ -843,6 +915,23 @@ static int uprobe_dispatcher(struct uprobe_consumer *con, struct pt_regs *regs)
         return ret;
 }
 
+static int uretprobe_dispatcher(struct uprobe_consumer *con,
+                                unsigned long func, struct pt_regs *regs)
+{
+        struct trace_uprobe *tu;
+
+        tu = container_of(con, struct trace_uprobe, consumer);
+
+        if (tu->flags & TP_FLAG_TRACE)
+                uretprobe_trace_func(tu, func, regs);
+
+#ifdef CONFIG_PERF_EVENTS
+        if (tu->flags & TP_FLAG_PROFILE)
+                uretprobe_perf_func(tu, func, regs);
+#endif
+        return 0;
+}
+
 static struct trace_event_functions uprobe_funcs = {
         .trace          = print_uprobe_event
 };
diff --git a/kernel/watchdog.c b/kernel/watchdog.c
index 4a944676358e..05039e348f07 100644
--- a/kernel/watchdog.c
+++ b/kernel/watchdog.c
@@ -517,6 +517,11 @@ int proc_dowatchdog(struct ctl_table *table, int write,
                 return ret;
 
         set_sample_period();
+        /*
+         * Watchdog threads shouldn't be enabled if they are
+         * disabled. The 'watchdog_disabled' variable check in
+         * watchdog_*_all_cpus() function takes care of this.
+         */
         if (watchdog_enabled && watchdog_thresh)
                 watchdog_enable_all_cpus();
         else
diff --git a/tools/Makefile b/tools/Makefile
index fa36565b209d..6aaeb6cd867d 100644
--- a/tools/Makefile
+++ b/tools/Makefile
@@ -34,7 +34,13 @@ help:
 cpupower: FORCE
         $(call descend,power/$@)
 
-cgroup firewire lguest perf usb virtio vm: FORCE
+cgroup firewire guest usb virtio vm: FORCE
+        $(call descend,$@)
+
+liblk: FORCE
+        $(call descend,lib/lk)
+
+perf: liblk FORCE
         $(call descend,$@)
 
 selftests: FORCE
@@ -62,7 +68,13 @@ install: cgroup_install cpupower_install firewire_install lguest_install \
 cpupower_clean:
         $(call descend,power/cpupower,clean)
 
-cgroup_clean firewire_clean lguest_clean perf_clean usb_clean virtio_clean vm_clean:
+cgroup_clean firewire_clean lguest_clean usb_clean virtio_clean vm_clean:
+        $(call descend,$(@:_clean=),clean)
+
+liblk_clean:
+        $(call descend,lib/lk,clean)
+
+perf_clean: liblk_clean
         $(call descend,$(@:_clean=),clean)
 
 selftests_clean:
diff --git a/tools/lib/lk/Makefile b/tools/lib/lk/Makefile
new file mode 100644
index 000000000000..926cbf3efc7f
--- /dev/null
+++ b/tools/lib/lk/Makefile
@@ -0,0 +1,35 @@
+include ../../scripts/Makefile.include
+
+# guard against environment variables
+LIB_H=
+LIB_OBJS=
+
+LIB_H += debugfs.h
+
+LIB_OBJS += $(OUTPUT)debugfs.o
+
+LIBFILE = liblk.a
+
+CFLAGS = -ggdb3 -Wall -Wextra -std=gnu99 -Werror -O6 -D_FORTIFY_SOURCE=2 $(EXTRA_WARNINGS) $(EXTRA_CFLAGS) -fPIC
+EXTLIBS = -lpthread -lrt -lelf -lm
+ALL_CFLAGS = $(CFLAGS) $(BASIC_CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
+ALL_LDFLAGS = $(LDFLAGS)
+
+RM = rm -f
+
+$(LIBFILE): $(LIB_OBJS)
+        $(QUIET_AR)$(RM) $@ && $(AR) rcs $(OUTPUT)$@ $(LIB_OBJS)
+
+$(LIB_OBJS): $(LIB_H)
+
+$(OUTPUT)%.o: %.c
+        $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
+$(OUTPUT)%.s: %.c
+        $(QUIET_CC)$(CC) -S $(ALL_CFLAGS) $<
+$(OUTPUT)%.o: %.S
+        $(QUIET_CC)$(CC) -o $@ -c $(ALL_CFLAGS) $<
+
+clean:
+        $(RM) $(LIB_OBJS) $(LIBFILE)
+
+.PHONY: clean
diff --git a/tools/perf/util/debugfs.c b/tools/lib/lk/debugfs.c
index dd8b19319c03..099e7cd022e4 100644
--- a/tools/perf/util/debugfs.c
+++ b/tools/lib/lk/debugfs.c
@@ -1,36 +1,39 @@
-#include "util.h"
-#include "debugfs.h"
-#include "cache.h"
-
-#include <linux/kernel.h>
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stdbool.h>
+#include <sys/vfs.h>
 #include <sys/mount.h>
+#include <linux/magic.h>
+#include <linux/kernel.h>
+
+#include "debugfs.h"
 
-static int debugfs_premounted;
 char debugfs_mountpoint[PATH_MAX + 1] = "/sys/kernel/debug";
-char tracing_events_path[PATH_MAX + 1] = "/sys/kernel/debug/tracing/events";
 
-static const char *debugfs_known_mountpoints[] = {
+static const char * const debugfs_known_mountpoints[] = {
         "/sys/kernel/debug/",
         "/debug/",
         0,
 };
 
-static int debugfs_found;
+static bool debugfs_found;
 
 /* find the path to the mounted debugfs */
 const char *debugfs_find_mountpoint(void)
 {
-        const char **ptr;
+        const char * const *ptr;
         char type[100];
         FILE *fp;
 
         if (debugfs_found)
-                return (const char *) debugfs_mountpoint;
+                return (const char *)debugfs_mountpoint;
 
         ptr = debugfs_known_mountpoints;
         while (*ptr) {
                 if (debugfs_valid_mountpoint(*ptr) == 0) {
-                        debugfs_found = 1;
+                        debugfs_found = true;
                         strcpy(debugfs_mountpoint, *ptr);
                         return debugfs_mountpoint;
                 }
@@ -52,7 +55,7 @@ const char *debugfs_find_mountpoint(void)
         if (strcmp(type, "debugfs") != 0)
                 return NULL;
 
-        debugfs_found = 1;
+        debugfs_found = true;
 
         return debugfs_mountpoint;
 }
@@ -71,21 +74,12 @@ int debugfs_valid_mountpoint(const char *debugfs)
         return 0;
 }
 
-static void debugfs_set_tracing_events_path(const char *mountpoint)
-{
-        snprintf(tracing_events_path, sizeof(tracing_events_path), "%s/%s",
-                 mountpoint, "tracing/events");
-}
-
 /* mount the debugfs somewhere if it's not mounted */
-
 char *debugfs_mount(const char *mountpoint)
 {
         /* see if it's already mounted */
-        if (debugfs_find_mountpoint()) {
-                debugfs_premounted = 1;
+        if (debugfs_find_mountpoint())
                 goto out;
-        }
 
         /* if not mounted and no argument */
         if (mountpoint == NULL) {
@@ -100,15 +94,8 @@ char *debugfs_mount(const char *mountpoint)
                 return NULL;
 
         /* save the mountpoint */
-        debugfs_found = 1;
+        debugfs_found = true;
         strncpy(debugfs_mountpoint, mountpoint, sizeof(debugfs_mountpoint));
 out:
-        debugfs_set_tracing_events_path(debugfs_mountpoint);
         return debugfs_mountpoint;
 }
-
-void debugfs_set_path(const char *mountpoint)
-{
-        snprintf(debugfs_mountpoint, sizeof(debugfs_mountpoint), "%s", mountpoint);
-        debugfs_set_tracing_events_path(mountpoint);
-}
diff --git a/tools/lib/lk/debugfs.h b/tools/lib/lk/debugfs.h
new file mode 100644
index 000000000000..935c59bdb442
--- /dev/null
+++ b/tools/lib/lk/debugfs.h
@@ -0,0 +1,29 @@
+#ifndef __LK_DEBUGFS_H__
+#define __LK_DEBUGFS_H__
+
+#define _STR(x) #x
+#define STR(x) _STR(x)
+
+/*
+ * On most systems <limits.h> would have given us this, but  not on some systems
+ * (e.g. GNU/Hurd).
+ */
+#ifndef PATH_MAX
+#define PATH_MAX 4096
+#endif
+
+#ifndef DEBUGFS_MAGIC
+#define DEBUGFS_MAGIC          0x64626720
+#endif
+
+#ifndef PERF_DEBUGFS_ENVIRONMENT
+#define PERF_DEBUGFS_ENVIRONMENT "PERF_DEBUGFS_DIR"
+#endif
+
+const char *debugfs_find_mountpoint(void);
+int debugfs_valid_mountpoint(const char *debugfs);
+char *debugfs_mount(const char *mountpoint);
+
+extern char debugfs_mountpoint[];
+
+#endif /* __LK_DEBUGFS_H__ */
diff --git a/tools/perf/Documentation/perf-annotate.txt b/tools/perf/Documentation/perf-annotate.txt
index 5ad07ef417f0..e9cd39a92dc2 100644
--- a/tools/perf/Documentation/perf-annotate.txt
+++ b/tools/perf/Documentation/perf-annotate.txt
@@ -93,6 +93,9 @@ OPTIONS
 --skip-missing::
         Skip symbols that cannot be annotated.
 
+--group::
+        Show event group information together
+
 SEE ALSO
 --------
 linkperf:perf-record[1], linkperf:perf-report[1]
diff --git a/tools/perf/Documentation/perf-mem.txt b/tools/perf/Documentation/perf-mem.txt
new file mode 100644
index 000000000000..888d51137fbe
--- /dev/null
+++ b/tools/perf/Documentation/perf-mem.txt
@@ -0,0 +1,48 @@
+perf-mem(1)
+===========
+
+NAME
+----
+perf-mem - Profile memory accesses
+
+SYNOPSIS
+--------
+[verse]
+'perf mem' [<options>] (record [<command>] | report)
+
+DESCRIPTION
+-----------
+"perf mem -t <TYPE> record" runs a command and gathers memory operation data
+from it, into perf.data. Perf record options are accepted and are passed through.
+
+"perf mem -t <TYPE> report" displays the result. It invokes perf report with the
+right set of options to display a memory access profile.
+
+OPTIONS
+-------
+<command>...::
+        Any command you can specify in a shell.
+
+-t::
+--type=::
+        Select the memory operation type: load or store (default: load)
+
+-D::
+--dump-raw-samples=::
+        Dump the raw decoded samples on the screen in a format that is easy to parse with
+        one sample per line.
+
+-x::
+--field-separator::
+        Specify the field separator used when dump raw samples (-D option). By default,
+        The separator is the space character.
+
+-C::
+--cpu-list::
+        Restrict dump of raw samples to those provided via this option. Note that the same
+        option can be passed in record mode. It will be interpreted the same way as perf
+        record.
+
+SEE ALSO
+--------
+linkperf:perf-record[1], linkperf:perf-report[1]
diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt
index 938e8904f64d..d4da111ef53d 100644
--- a/tools/perf/Documentation/perf-record.txt
+++ b/tools/perf/Documentation/perf-record.txt
@@ -182,6 +182,12 @@ is enabled for all the sampling events. The sampled branch type is the same for
 The various filters must be specified as a comma separated list: --branch-filter any_ret,u,k
 Note that this feature may not be available on all processors.
 
+-W::
+--weight::
+Enable weightened sampling. An additional weight is recorded per sample and can be
+displayed with the weight and local_weight sort keys.  This currently works for TSX
+abort events and some memory events in precise mode on modern Intel CPUs.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-list[1]
diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt
index 02284a0067f0..7d5f4f38aa52 100644
--- a/tools/perf/Documentation/perf-report.txt
+++ b/tools/perf/Documentation/perf-report.txt
@@ -59,7 +59,7 @@ OPTIONS
 --sort=::
         Sort histogram entries by given key(s) - multiple keys can be specified
         in CSV format.  Following sort keys are available:
-        pid, comm, dso, symbol, parent, cpu, srcline.
+        pid, comm, dso, symbol, parent, cpu, srcline, weight, local_weight.
 
         Each key has following meaning:
 
@@ -206,6 +206,10 @@ OPTIONS
 --group::
         Show event group information together.
 
+--demangle::
+        Demangle symbol names to human readable form. It's enabled by default,
+        disable with --no-demangle.
+
 SEE ALSO
 --------
 linkperf:perf-stat[1], linkperf:perf-annotate[1]
diff --git a/tools/perf/Documentation/perf-stat.txt b/tools/perf/Documentation/perf-stat.txt
index faf4f4feebcc..2fe87fb558f0 100644
--- a/tools/perf/Documentation/perf-stat.txt
+++ b/tools/perf/Documentation/perf-stat.txt
@@ -52,7 +52,7 @@ OPTIONS
 
 -r::
 --repeat=<n>::
-        repeat command and print average + stddev (max: 100)
+        repeat command and print average + stddev (max: 100). 0 means forever.
 
 -B::
 --big-num::
@@ -119,13 +119,19 @@ perf stat --repeat 10 --null --sync --pre 'make -s O=defconfig-build/clean' -- m
         Print count deltas every N milliseconds (minimum: 100ms)
         example: perf stat -I 1000 -e cycles -a sleep 5
 
---aggr-socket::
+--per-socket::
 Aggregate counts per processor socket for system-wide mode measurements.  This
 is a useful mode to detect imbalance between sockets.  To enable this mode,
-use --aggr-socket in addition to -a. (system-wide).  The output includes the
+use --per-socket in addition to -a. (system-wide).  The output includes the
 socket number and the number of online processors on that socket. This is
 useful to gauge the amount of aggregation.
 
+--per-core::
+Aggregate counts per physical processor for system-wide mode measurements.  This
+is a useful mode to detect imbalance between physical cores.  To enable this mode,
+use --per-core in addition to -a. (system-wide).  The output includes the
+core number and the number of online logical processors on that physical processor.
+
 EXAMPLES
 --------
 
diff --git a/tools/perf/Documentation/perf-top.txt b/tools/perf/Documentation/perf-top.txt
index a414bc95fd52..9f1a2fe54757 100644
--- a/tools/perf/Documentation/perf-top.txt
+++ b/tools/perf/Documentation/perf-top.txt
@@ -112,7 +112,7 @@ Default is to monitor all CPUS.
 
 -s::
 --sort::
-        Sort by key(s): pid, comm, dso, symbol, parent, srcline.
+        Sort by key(s): pid, comm, dso, symbol, parent, srcline, weight, local_weight.
 
 -n::
 --show-nr-samples::
diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index 39d41068484f..025de796067c 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -1,6 +1,7 @@
 tools/perf
 tools/scripts
 tools/lib/traceevent
+tools/lib/lk
 include/linux/const.h
 include/linux/perf_event.h
 include/linux/rbtree.h
diff --git a/tools/perf/Makefile b/tools/perf/Makefile
index bb74c79cd16e..b0f164b133d9 100644
--- a/tools/perf/Makefile
+++ b/tools/perf/Makefile
@@ -35,7 +35,9 @@ include config/utilities.mak
 #
 # Define WERROR=0 to disable treating any warnings as errors.
 #
-# Define NO_NEWT if you do not want TUI support.
+# Define NO_NEWT if you do not want TUI support. (deprecated)
+#
+# Define NO_SLANG if you do not want TUI support.
 #
 # Define NO_GTK2 if you do not want GTK+ GUI support.
 #
@@ -104,6 +106,10 @@ ifdef PARSER_DEBUG
         PARSER_DEBUG_CFLAGS := -DPARSER_DEBUG
 endif
 
+ifdef NO_NEWT
+        NO_SLANG=1
+endif
+
 CFLAGS = -fno-omit-frame-pointer -ggdb3 -funwind-tables -Wall -Wextra -std=gnu99 $(CFLAGS_WERROR) $(CFLAGS_OPTIMIZE) $(EXTRA_WARNINGS) $(EXTRA_CFLAGS) $(PARSER_DEBUG_CFLAGS)
 EXTLIBS = -lpthread -lrt -lelf -lm
 ALL_CFLAGS = $(CFLAGS) -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
@@ -215,6 +221,7 @@ BASIC_CFLAGS = \
         -Iutil \
         -I. \
         -I$(TRACE_EVENT_DIR) \
+        -I../lib/ \
         -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE
 
 BASIC_LDFLAGS =
@@ -240,19 +247,28 @@ SCRIPT_SH += perf-archive.sh
 grep-libs = $(filter -l%,$(1))
 strip-libs = $(filter-out -l%,$(1))
 
+LK_DIR = ../lib/lk/
 TRACE_EVENT_DIR = ../lib/traceevent/
 
+LK_PATH=$(LK_DIR)
+
 ifneq ($(OUTPUT),)
         TE_PATH=$(OUTPUT)
+ifneq ($(subdir),)
+        LK_PATH=$(OUTPUT)$(LK_DIR)
+else
+        LK_PATH=$(OUTPUT)
+endif
 else
         TE_PATH=$(TRACE_EVENT_DIR)
 endif
 
 LIBTRACEEVENT = $(TE_PATH)libtraceevent.a
-TE_LIB := -L$(TE_PATH) -ltraceevent
-
 export LIBTRACEEVENT
 
+LIBLK = $(LK_PATH)liblk.a
+export LIBLK
+
 # python extension build directories
 PYTHON_EXTBUILD     := $(OUTPUT)python_ext_build/
 PYTHON_EXTBUILD_LIB := $(PYTHON_EXTBUILD)lib/
@@ -262,7 +278,7 @@ export PYTHON_EXTBUILD_LIB PYTHON_EXTBUILD_TMP
 python-clean := rm -rf $(PYTHON_EXTBUILD) $(OUTPUT)python/perf.so
 
 PYTHON_EXT_SRCS := $(shell grep -v ^\# util/python-ext-sources)
-PYTHON_EXT_DEPS := util/python-ext-sources util/setup.py
+PYTHON_EXT_DEPS := util/python-ext-sources util/setup.py $(LIBTRACEEVENT)
 
 $(OUTPUT)python/perf.so: $(PYTHON_EXT_SRCS) $(PYTHON_EXT_DEPS)
         $(QUIET_GEN)CFLAGS='$(BASIC_CFLAGS)' $(PYTHON_WORD) util/setup.py \
@@ -355,7 +371,6 @@ LIB_H += util/cache.h
 LIB_H += util/callchain.h
 LIB_H += util/build-id.h
 LIB_H += util/debug.h
-LIB_H += util/debugfs.h
 LIB_H += util/sysfs.h
 LIB_H += util/pmu.h
 LIB_H += util/event.h
@@ -416,7 +431,6 @@ LIB_OBJS += $(OUTPUT)util/annotate.o
 LIB_OBJS += $(OUTPUT)util/build-id.o
 LIB_OBJS += $(OUTPUT)util/config.o
 LIB_OBJS += $(OUTPUT)util/ctype.o
-LIB_OBJS += $(OUTPUT)util/debugfs.o
 LIB_OBJS += $(OUTPUT)util/sysfs.o
 LIB_OBJS += $(OUTPUT)util/pmu.o
 LIB_OBJS += $(OUTPUT)util/environment.o
@@ -503,6 +517,10 @@ LIB_OBJS += $(OUTPUT)tests/evsel-tp-sched.o
 LIB_OBJS += $(OUTPUT)tests/pmu.o
 LIB_OBJS += $(OUTPUT)tests/hists_link.o
 LIB_OBJS += $(OUTPUT)tests/python-use.o
+LIB_OBJS += $(OUTPUT)tests/bp_signal.o
+LIB_OBJS += $(OUTPUT)tests/bp_signal_overflow.o
+LIB_OBJS += $(OUTPUT)tests/task-exit.o
+LIB_OBJS += $(OUTPUT)tests/sw-clock.o
 
 BUILTIN_OBJS += $(OUTPUT)builtin-annotate.o
 BUILTIN_OBJS += $(OUTPUT)builtin-bench.o
@@ -535,8 +553,9 @@ BUILTIN_OBJS += $(OUTPUT)builtin-lock.o
 BUILTIN_OBJS += $(OUTPUT)builtin-kvm.o
 BUILTIN_OBJS += $(OUTPUT)builtin-inject.o
 BUILTIN_OBJS += $(OUTPUT)tests/builtin-test.o
+BUILTIN_OBJS += $(OUTPUT)builtin-mem.o
 
-PERFLIBS = $(LIB_FILE) $(LIBTRACEEVENT)
+PERFLIBS = $(LIB_FILE) $(LIBLK) $(LIBTRACEEVENT)
 
 #
 # Platform specific tweaks
@@ -667,15 +686,15 @@ ifndef NO_LIBAUDIT
         endif
 endif
 
-ifndef NO_NEWT
-        FLAGS_NEWT=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) -lnewt
-        ifneq ($(call try-cc,$(SOURCE_NEWT),$(FLAGS_NEWT),libnewt),y)
-                msg := $(warning newt not found, disables TUI support. Please install newt-devel or libnewt-dev);
+ifndef NO_SLANG
+        FLAGS_SLANG=$(ALL_CFLAGS) $(ALL_LDFLAGS) $(EXTLIBS) -I/usr/include/slang -lslang
+        ifneq ($(call try-cc,$(SOURCE_SLANG),$(FLAGS_SLANG),libslang),y)
+                msg := $(warning slang not found, disables TUI support. Please install slang-devel or libslang-dev);
         else
                 # Fedora has /usr/include/slang/slang.h, but ubuntu /usr/include/slang.h
                 BASIC_CFLAGS += -I/usr/include/slang
-                BASIC_CFLAGS += -DNEWT_SUPPORT
-                EXTLIBS += -lnewt -lslang
+                BASIC_CFLAGS += -DSLANG_SUPPORT
+                EXTLIBS += -lslang
                 LIB_OBJS += $(OUTPUT)ui/browser.o
                 LIB_OBJS += $(OUTPUT)ui/browsers/annotate.o
                 LIB_OBJS += $(OUTPUT)ui/browsers/hists.o
@@ -1051,6 +1070,18 @@ $(LIBTRACEEVENT):
 $(LIBTRACEEVENT)-clean:
         $(QUIET_SUBDIR0)$(TRACE_EVENT_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) clean
 
+# if subdir is set, we've been called from above so target has been built
+# already
+$(LIBLK):
+ifeq ($(subdir),)
+        $(QUIET_SUBDIR0)$(LK_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) liblk.a
+endif
+
+$(LIBLK)-clean:
+ifeq ($(subdir),)
+        $(QUIET_SUBDIR0)$(LK_DIR) $(QUIET_SUBDIR1) O=$(OUTPUT) clean
+endif
+
 help:
         @echo 'Perf make targets:'
         @echo '  doc            - make *all* documentation (see below)'
@@ -1171,7 +1202,7 @@ $(INSTALL_DOC_TARGETS):
 
 ### Cleaning rules
 
-clean: $(LIBTRACEEVENT)-clean
+clean: $(LIBTRACEEVENT)-clean $(LIBLK)-clean
         $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf.o $(LANG_BINDINGS)
         $(RM) $(ALL_PROGRAMS) perf
         $(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope*
@@ -1181,6 +1212,6 @@ clean: $(LIBTRACEEVENT)-clean
         $(RM) $(OUTPUT)util/*-flex*
         $(python-clean)
 
-.PHONY: all install clean strip $(LIBTRACEEVENT)
+.PHONY: all install clean strip $(LIBTRACEEVENT) $(LIBLK)
 .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell
 .PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope .FORCE-PERF-CFLAGS
diff --git a/tools/perf/arch/arm/util/dwarf-regs.c b/tools/perf/arch/arm/util/dwarf-regs.c
index e8d5c551c69c..33ec5b339da8 100644
--- a/tools/perf/arch/arm/util/dwarf-regs.c
+++ b/tools/perf/arch/arm/util/dwarf-regs.c
@@ -8,10 +8,7 @@
  * published by the Free Software Foundation.
  */
 
-#include <stdlib.h>
-#ifndef __UCLIBC__
-#include <libio.h>
-#endif
+#include <stddef.h>
 #include <dwarf-regs.h>
 
 struct pt_regs_dwarfnum {
diff --git a/tools/perf/arch/powerpc/util/dwarf-regs.c b/tools/perf/arch/powerpc/util/dwarf-regs.c
index 7cdd61d0e27c..733151cdf46e 100644
--- a/tools/perf/arch/powerpc/util/dwarf-regs.c
+++ b/tools/perf/arch/powerpc/util/dwarf-regs.c
@@ -9,10 +9,7 @@
  * 2 of the License, or (at your option) any later version.
  */
 
-#include <stdlib.h>
-#ifndef __UCLIBC__
-#include <libio.h>
-#endif
+#include <stddef.h>
 #include <dwarf-regs.h>
 
 
diff --git a/tools/perf/arch/s390/util/dwarf-regs.c b/tools/perf/arch/s390/util/dwarf-regs.c
index e19653e025fa..0469df02ee62 100644
--- a/tools/perf/arch/s390/util/dwarf-regs.c
+++ b/tools/perf/arch/s390/util/dwarf-regs.c
@@ -6,7 +6,7 @@
  *
  */
 
-#include <libio.h>
+#include <stddef.h>
 #include <dwarf-regs.h>
 
 #define NUM_GPRS 16
diff --git a/tools/perf/arch/sh/util/dwarf-regs.c b/tools/perf/arch/sh/util/dwarf-regs.c
index a11edb007a6c..0d0897f57a10 100644
--- a/tools/perf/arch/sh/util/dwarf-regs.c
+++ b/tools/perf/arch/sh/util/dwarf-regs.c
@@ -19,7 +19,7 @@
  *
  */
 
-#include <libio.h>
+#include <stddef.h>
 #include <dwarf-regs.h>
 
 /*
diff --git a/tools/perf/arch/sparc/util/dwarf-regs.c b/tools/perf/arch/sparc/util/dwarf-regs.c
index 0ab88483720c..92eda412fed3 100644
--- a/tools/perf/arch/sparc/util/dwarf-regs.c
+++ b/tools/perf/arch/sparc/util/dwarf-regs.c
@@ -9,7 +9,7 @@
  * 2 of the License, or (at your option) any later version.
  */
 
-#include <libio.h>
+#include <stddef.h>
 #include <dwarf-regs.h>
 
 #define SPARC_MAX_REGS  96
diff --git a/tools/perf/arch/x86/util/dwarf-regs.c b/tools/perf/arch/x86/util/dwarf-regs.c
index a794d3081928..be22dd463232 100644
--- a/tools/perf/arch/x86/util/dwarf-regs.c
+++ b/tools/perf/arch/x86/util/dwarf-regs.c
@@ -20,7 +20,7 @@
  *
  */
 
-#include <libio.h>
+#include <stddef.h>
 #include <dwarf-regs.h>
 
 /*
diff --git a/tools/perf/builtin-annotate.c b/tools/perf/builtin-annotate.c
index 2e6961ea3184..db491e9a812b 100644
--- a/tools/perf/builtin-annotate.c
+++ b/tools/perf/builtin-annotate.c
@@ -63,7 +63,7 @@ static int perf_evsel__add_sample(struct perf_evsel *evsel,
                 return 0;
         }
 
-        he = __hists__add_entry(&evsel->hists, al, NULL, 1);
+        he = __hists__add_entry(&evsel->hists, al, NULL, 1, 1);
         if (he == NULL)
                 return -ENOMEM;
 
@@ -109,14 +109,16 @@ static int process_sample_event(struct perf_tool *tool,
         return 0;
 }
 
-static int hist_entry__tty_annotate(struct hist_entry *he, int evidx,
+static int hist_entry__tty_annotate(struct hist_entry *he,
+                                    struct perf_evsel *evsel,
                                     struct perf_annotate *ann)
 {
-        return symbol__tty_annotate(he->ms.sym, he->ms.map, evidx,
+        return symbol__tty_annotate(he->ms.sym, he->ms.map, evsel,
                                     ann->print_line, ann->full_paths, 0, 0);
 }
 
-static void hists__find_annotations(struct hists *self, int evidx,
+static void hists__find_annotations(struct hists *self,
+                                    struct perf_evsel *evsel,
                                     struct perf_annotate *ann)
 {
         struct rb_node *nd = rb_first(&self->entries), *next;
@@ -142,14 +144,14 @@ find_next:
                 if (use_browser == 2) {
                         int ret;
 
-                        ret = hist_entry__gtk_annotate(he, evidx, NULL);
+                        ret = hist_entry__gtk_annotate(he, evsel, NULL);
                         if (!ret || !ann->skip_missing)
                                 return;
 
                         /* skip missing symbols */
                         nd = rb_next(nd);
                 } else if (use_browser == 1) {
-                        key = hist_entry__tui_annotate(he, evidx, NULL);
+                        key = hist_entry__tui_annotate(he, evsel, NULL);
                         switch (key) {
                         case -1:
                                 if (!ann->skip_missing)
@@ -168,7 +170,7 @@ find_next:
                         if (next != NULL)
                                 nd = next;
                 } else {
-                        hist_entry__tty_annotate(he, evidx, ann);
+                        hist_entry__tty_annotate(he, evsel, ann);
                         nd = rb_next(nd);
                         /*
                          * Since we have a hist_entry per IP for the same
@@ -230,7 +232,12 @@ static int __cmd_annotate(struct perf_annotate *ann)
                         total_nr_samples += nr_samples;
                         hists__collapse_resort(hists);
                         hists__output_resort(hists);
-                        hists__find_annotations(hists, pos->idx, ann);
+
+                        if (symbol_conf.event_group &&
+                            !perf_evsel__is_group_leader(pos))
+                                continue;
+
+                        hists__find_annotations(hists, pos, ann);
                 }
         }
 
@@ -312,6 +319,8 @@ int cmd_annotate(int argc, const char **argv, const char *prefix __maybe_unused)
                    "Specify disassembler style (e.g. -M intel for intel syntax)"),
         OPT_STRING(0, "objdump", &objdump_path, "path",
                    "objdump binary to use for disassembly and annotations"),
+        OPT_BOOLEAN(0, "group", &symbol_conf.event_group,
+                    "Show event group information together"),
         OPT_END()
         };
 
diff --git a/tools/perf/builtin-diff.c b/tools/perf/builtin-diff.c
index d207a97a2db1..2d0462d89a97 100644
--- a/tools/perf/builtin-diff.c
+++ b/tools/perf/builtin-diff.c
@@ -231,9 +231,10 @@ int perf_diff__formula(struct hist_entry *he, struct hist_entry *pair,
 }
 
 static int hists__add_entry(struct hists *self,
-                            struct addr_location *al, u64 period)
+                            struct addr_location *al, u64 period,
+                            u64 weight)
 {
-        if (__hists__add_entry(self, al, NULL, period) != NULL)
+        if (__hists__add_entry(self, al, NULL, period, weight) != NULL)
                 return 0;
         return -ENOMEM;
 }
@@ -255,7 +256,7 @@ static int diff__process_sample_event(struct perf_tool *tool __maybe_unused,
         if (al.filtered)
                 return 0;
 
-        if (hists__add_entry(&evsel->hists, &al, sample->period)) {
+        if (hists__add_entry(&evsel->hists, &al, sample->period, sample->weight)) {
                 pr_warning("problem incrementing symbol period, skipping event\n");
                 return -1;
         }
diff --git a/tools/perf/builtin-kvm.c b/tools/perf/builtin-kvm.c
index 37a769d7f9fe..533501e2b07c 100644
--- a/tools/perf/builtin-kvm.c
+++ b/tools/perf/builtin-kvm.c
@@ -12,7 +12,7 @@
 #include "util/parse-options.h"
 #include "util/trace-event.h"
 #include "util/debug.h"
-#include "util/debugfs.h"
+#include <lk/debugfs.h>
 #include "util/tool.h"
 #include "util/stat.h"
 
diff --git a/tools/perf/builtin-mem.c b/tools/perf/builtin-mem.c
new file mode 100644
index 000000000000..a8ff6d264e50
--- /dev/null
+++ b/tools/perf/builtin-mem.c
@@ -0,0 +1,242 @@
+#include "builtin.h"
+#include "perf.h"
+
+#include "util/parse-options.h"
+#include "util/trace-event.h"
+#include "util/tool.h"
+#include "util/session.h"
+
+#define MEM_OPERATION_LOAD      "load"
+#define MEM_OPERATION_STORE     "store"
+
+static const char       *mem_operation          = MEM_OPERATION_LOAD;
+
+struct perf_mem {
+        struct perf_tool        tool;
+        char const              *input_name;
+        symbol_filter_t         annotate_init;
+        bool                    hide_unresolved;
+        bool                    dump_raw;
+        const char              *cpu_list;
+        DECLARE_BITMAP(cpu_bitmap, MAX_NR_CPUS);
+};
+
+static const char * const mem_usage[] = {
+        "perf mem [<options>] {record <command> |report}",
+        NULL
+};
+
+static int __cmd_record(int argc, const char **argv)
+{
+        int rec_argc, i = 0, j;
+        const char **rec_argv;
+        char event[64];
+        int ret;
+
+        rec_argc = argc + 4;
+        rec_argv = calloc(rec_argc + 1, sizeof(char *));
+        if (!rec_argv)
+                return -1;
+
+        rec_argv[i++] = strdup("record");
+        if (!strcmp(mem_operation, MEM_OPERATION_LOAD))
+                rec_argv[i++] = strdup("-W");
+        rec_argv[i++] = strdup("-d");
+        rec_argv[i++] = strdup("-e");
+
+        if (strcmp(mem_operation, MEM_OPERATION_LOAD))
+                sprintf(event, "cpu/mem-stores/pp");
+        else
+                sprintf(event, "cpu/mem-loads/pp");
+
+        rec_argv[i++] = strdup(event);
+        for (j = 1; j < argc; j++, i++)
+                rec_argv[i] = argv[j];
+
+        ret = cmd_record(i, rec_argv, NULL);
+        free(rec_argv);
+        return ret;
+}
+
+static int
+dump_raw_samples(struct perf_tool *tool,
+                 union perf_event *event,
+                 struct perf_sample *sample,
+                 struct perf_evsel *evsel __maybe_unused,
+                 struct machine *machine)
+{
+        struct perf_mem *mem = container_of(tool, struct perf_mem, tool);
+        struct addr_location al;
+        const char *fmt;
+
+        if (perf_event__preprocess_sample(event, machine, &al, sample,
+                                mem->annotate_init) < 0) {
+                fprintf(stderr, "problem processing %d event, skipping it.\n",
+                                event->header.type);
+                return -1;
+        }
+
+        if (al.filtered || (mem->hide_unresolved && al.sym == NULL))
+                return 0;
+
+        if (al.map != NULL)
+                al.map->dso->hit = 1;
+
+        if (symbol_conf.field_sep) {
+                fmt = "%d%s%d%s0x%"PRIx64"%s0x%"PRIx64"%s%"PRIu64
+                      "%s0x%"PRIx64"%s%s:%s\n";
+        } else {
+                fmt = "%5d%s%5d%s0x%016"PRIx64"%s0x016%"PRIx64
+                      "%s%5"PRIu64"%s0x%06"PRIx64"%s%s:%s\n";
+                symbol_conf.field_sep = " ";
+        }
+
+        printf(fmt,
+                sample->pid,
+                symbol_conf.field_sep,
+                sample->tid,
+                symbol_conf.field_sep,
+                event->ip.ip,
+                symbol_conf.field_sep,
+                sample->addr,
+                symbol_conf.field_sep,
+                sample->weight,
+                symbol_conf.field_sep,
+                sample->data_src,
+                symbol_conf.field_sep,
+                al.map ? (al.map->dso ? al.map->dso->long_name : "???") : "???",
+                al.sym ? al.sym->name : "???");
+
+        return 0;
+}
+
+static int process_sample_event(struct perf_tool *tool,
+                                union perf_event *event,
+                                struct perf_sample *sample,
+                                struct perf_evsel *evsel,
+                                struct machine *machine)
+{
+        return dump_raw_samples(tool, event, sample, evsel, machine);
+}
+
+static int report_raw_events(struct perf_mem *mem)
+{
+        int err = -EINVAL;
+        int ret;
+        struct perf_session *session = perf_session__new(input_name, O_RDONLY,
+                                                         0, false, &mem->tool);
+
+        if (session == NULL)
+                return -ENOMEM;
+
+        if (mem->cpu_list) {
+                ret = perf_session__cpu_bitmap(session, mem->cpu_list,
+                                               mem->cpu_bitmap);
+                if (ret)
+                        goto out_delete;
+        }
+
+        if (symbol__init() < 0)
+                return -1;
+
+        printf("# PID, TID, IP, ADDR, LOCAL WEIGHT, DSRC, SYMBOL\n");
+
+        err = perf_session__process_events(session, &mem->tool);
+        if (err)
+                return err;
+
+        return 0;
+
+out_delete:
+        perf_session__delete(session);
+        return err;
+}
+
+static int report_events(int argc, const char **argv, struct perf_mem *mem)
+{
+        const char **rep_argv;
+        int ret, i = 0, j, rep_argc;
+
+        if (mem->dump_raw)
+                return report_raw_events(mem);
+
+        rep_argc = argc + 3;
+        rep_argv = calloc(rep_argc + 1, sizeof(char *));
+        if (!rep_argv)
+                return -1;
+
+        rep_argv[i++] = strdup("report");
+        rep_argv[i++] = strdup("--mem-mode");
+        rep_argv[i++] = strdup("-n"); /* display number of samples */
+
+        /*
+         * there is no weight (cost) associated with stores, so don't print
+         * the column
+         */
+        if (strcmp(mem_operation, MEM_OPERATION_LOAD))
+                rep_argv[i++] = strdup("--sort=mem,sym,dso,symbol_daddr,"
+                                       "dso_daddr,tlb,locked");
+
+        for (j = 1; j < argc; j++, i++)
+                rep_argv[i] = argv[j];
+
+        ret = cmd_report(i, rep_argv, NULL);
+        free(rep_argv);
+        return ret;
+}
+
+int cmd_mem(int argc, const char **argv, const char *prefix __maybe_unused)
+{
+        struct stat st;
+        struct perf_mem mem = {
+                .tool = {
+                        .sample         = process_sample_event,
+                        .mmap           = perf_event__process_mmap,
+                        .comm           = perf_event__process_comm,
+                        .lost           = perf_event__process_lost,
+                        .fork           = perf_event__process_fork,
+                        .build_id       = perf_event__process_build_id,
+                        .ordered_samples = true,
+                },
+                .input_name              = "perf.data",
+        };
+        const struct option mem_options[] = {
+        OPT_STRING('t', "type", &mem_operation,
+                   "type", "memory operations(load/store)"),
+        OPT_BOOLEAN('D', "dump-raw-samples", &mem.dump_raw,
+                    "dump raw samples in ASCII"),
+        OPT_BOOLEAN('U', "hide-unresolved", &mem.hide_unresolved,
+                    "Only display entries resolved to a symbol"),
+        OPT_STRING('i', "input", &input_name, "file",
+                   "input file name"),
+        OPT_STRING('C', "cpu", &mem.cpu_list, "cpu",
+                   "list of cpus to profile"),
+        OPT_STRING('x', "field-separator", &symbol_conf.field_sep,
+                   "separator",
+                   "separator for columns, no spaces will be added"
+                   " between columns '.' is reserved."),
+        OPT_END()
+        };
+
+        argc = parse_options(argc, argv, mem_options, mem_usage,
+                             PARSE_OPT_STOP_AT_NON_OPTION);
+
+        if (!argc || !(strncmp(argv[0], "rec", 3) || mem_operation))
+                usage_with_options(mem_usage, mem_options);
+
+        if (!mem.input_name || !strlen(mem.input_name)) {
+                if (!fstat(STDIN_FILENO, &st) && S_ISFIFO(st.st_mode))
+                        mem.input_name = "-";
+                else
+                        mem.input_name = "perf.data";
+        }
+
+        if (!strncmp(argv[0], "rec", 3))
+                return __cmd_record(argc, argv);
+        else if (!strncmp(argv[0], "rep", 3))
+                return report_events(argc, argv, &mem);
+        else
+                usage_with_options(mem_usage, mem_options);
+
+        return 0;
+}
diff --git a/tools/perf/builtin-probe.c b/tools/perf/builtin-probe.c
index de38a034b129..e8a66f9a6715 100644
--- a/tools/perf/builtin-probe.c
+++ b/tools/perf/builtin-probe.c
@@ -37,7 +37,7 @@
 #include "util/strfilter.h"
 #include "util/symbol.h"
 #include "util/debug.h"
-#include "util/debugfs.h"
+#include <lk/debugfs.h>
 #include "util/parse-options.h"
 #include "util/probe-finder.h"
 #include "util/probe-event.h"
diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index f1a939ebc19c..cdf58ecc04b1 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -5,8 +5,6 @@
  * (or a CPU, or a PID) into the perf.data output file - for
  * later analysis via perf report.
  */
-#define _FILE_OFFSET_BITS 64
-
 #include "builtin.h"
 
 #include "perf.h"
@@ -474,7 +472,9 @@ static int __cmd_record(struct perf_record *rec, int argc, const char **argv)
         }
 
         if (forks) {
-                err = perf_evlist__prepare_workload(evsel_list, opts, argv);
+                err = perf_evlist__prepare_workload(evsel_list, &opts->target,
+                                                    argv, opts->pipe_output,
+                                                    true);
                 if (err < 0) {
                         pr_err("Couldn't run the workload!\n");
                         goto out_delete_session;
@@ -953,6 +953,8 @@ const struct option record_options[] = {
         OPT_CALLBACK('j', "branch-filter", &record.opts.branch_stack,
                      "branch filter mask", "branch stack filter modes",
                      parse_branch_stack),
+        OPT_BOOLEAN('W', "weight", &record.opts.sample_weight,
+                    "sample by weight (on special events only)"),
         OPT_END()
 };
 
@@ -964,7 +966,7 @@ int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
         struct perf_record *rec = &record;
         char errbuf[BUFSIZ];
 
-        evsel_list = perf_evlist__new(NULL, NULL);
+        evsel_list = perf_evlist__new();
         if (evsel_list == NULL)
                 return -ENOMEM;
 
@@ -1026,7 +1028,7 @@ int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
                 ui__error("%s", errbuf);
 
                 err = -saved_errno;
-                goto out_free_fd;
+                goto out_symbol_exit;
         }
 
         err = -ENOMEM;
@@ -1057,6 +1059,9 @@ int cmd_record(int argc, const char **argv, const char *prefix __maybe_unused)
         }
 
         err = __cmd_record(&record, argc, argv);
+
+        perf_evlist__munmap(evsel_list);
+        perf_evlist__close(evsel_list);
 out_free_fd:
         perf_evlist__delete_maps(evsel_list);
 out_symbol_exit:
diff --git a/tools/perf/builtin-report.c b/tools/perf/builtin-report.c
index 96b5a7fee4bb..bd0ca81eeaca 100644
--- a/tools/perf/builtin-report.c
+++ b/tools/perf/builtin-report.c
@@ -13,7 +13,6 @@
 #include "util/annotate.h"
 #include "util/color.h"
 #include <linux/list.h>
-#include "util/cache.h"
 #include <linux/rbtree.h>
 #include "util/symbol.h"
 #include "util/callchain.h"
@@ -47,6 +46,7 @@ struct perf_report {
         bool                    show_full_info;
         bool                    show_threads;
         bool                    inverted_callchain;
+        bool                    mem_mode;
         struct perf_read_values show_threads_values;
         const char              *pretty_printing_style;
         symbol_filter_t         annotate_init;
@@ -65,6 +65,99 @@ static int perf_report_config(const char *var, const char *value, void *cb)
         return perf_default_config(var, value, cb);
 }
 
+static int perf_report__add_mem_hist_entry(struct perf_tool *tool,
+                                           struct addr_location *al,
+                                           struct perf_sample *sample,
+                                           struct perf_evsel *evsel,
+                                           struct machine *machine,
+                                           union perf_event *event)
+{
+        struct perf_report *rep = container_of(tool, struct perf_report, tool);
+        struct symbol *parent = NULL;
+        u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
+        int err = 0;
+        struct hist_entry *he;
+        struct mem_info *mi, *mx;
+        uint64_t cost;
+
+        if ((sort__has_parent || symbol_conf.use_callchain) &&
+            sample->callchain) {
+                err = machine__resolve_callchain(machine, evsel, al->thread,
+                                                 sample, &parent);
+                if (err)
+                        return err;
+        }
+
+        mi = machine__resolve_mem(machine, al->thread, sample, cpumode);
+        if (!mi)
+                return -ENOMEM;
+
+        if (rep->hide_unresolved && !al->sym)
+                return 0;
+
+        cost = sample->weight;
+        if (!cost)
+                cost = 1;
+
+        /*
+         * must pass period=weight in order to get the correct
+         * sorting from hists__collapse_resort() which is solely
+         * based on periods. We want sorting be done on nr_events * weight
+         * and this is indirectly achieved by passing period=weight here
+         * and the he_stat__add_period() function.
+         */
+        he = __hists__add_mem_entry(&evsel->hists, al, parent, mi, cost, cost);
+        if (!he)
+                return -ENOMEM;
+
+        /*
+         * In the TUI browser, we are doing integrated annotation,
+         * so we don't allocate the extra space needed because the stdio
+         * code will not use it.
+         */
+        if (sort__has_sym && he->ms.sym && use_browser > 0) {
+                struct annotation *notes = symbol__annotation(he->ms.sym);
+
+                assert(evsel != NULL);
+
+                if (notes->src == NULL && symbol__alloc_hist(he->ms.sym) < 0)
+                        goto out;
+
+                err = hist_entry__inc_addr_samples(he, evsel->idx, al->addr);
+                if (err)
+                        goto out;
+        }
+
+        if (sort__has_sym && he->mem_info->daddr.sym && use_browser > 0) {
+                struct annotation *notes;
+
+                mx = he->mem_info;
+
+                notes = symbol__annotation(mx->daddr.sym);
+                if (notes->src == NULL && symbol__alloc_hist(mx->daddr.sym) < 0)
+                        goto out;
+
+                err = symbol__inc_addr_samples(mx->daddr.sym,
+                                               mx->daddr.map,
+                                               evsel->idx,
+                                               mx->daddr.al_addr);
+                if (err)
+                        goto out;
+        }
+
+        evsel->hists.stats.total_period += cost;
+        hists__inc_nr_events(&evsel->hists, PERF_RECORD_SAMPLE);
+        err = 0;
+
+        if (symbol_conf.use_callchain) {
+                err = callchain_append(he->callchain,
+                                       &callchain_cursor,
+                                       sample->period);
+        }
+out:
+        return err;
+}
+
 static int perf_report__add_branch_hist_entry(struct perf_tool *tool,
                                         struct addr_location *al,
                                         struct perf_sample *sample,
@@ -99,7 +192,7 @@ static int perf_report__add_branch_hist_entry(struct perf_tool *tool,
                  * and not events sampled. Thus we use a pseudo period of 1.
                  */
                 he = __hists__add_branch_entry(&evsel->hists, al, parent,
-                                &bi[i], 1);
+                                &bi[i], 1, 1);
                 if (he) {
                         struct annotation *notes;
                         err = -ENOMEM;
@@ -157,7 +250,8 @@ static int perf_evsel__add_hist_entry(struct perf_evsel *evsel,
                         return err;
         }
 
-        he = __hists__add_entry(&evsel->hists, al, parent, sample->period);
+        he = __hists__add_entry(&evsel->hists, al, parent, sample->period,
+                                        sample->weight);
         if (he == NULL)
                 return -ENOMEM;
 
@@ -169,7 +263,7 @@ static int perf_evsel__add_hist_entry(struct perf_evsel *evsel,
                         return err;
         }
         /*
-         * Only in the newt browser we are doing integrated annotation,
+         * Only in the TUI browser we are doing integrated annotation,
          * so we don't allocated the extra space needed because the stdio
          * code will not use it.
          */
@@ -220,6 +314,12 @@ static int process_sample_event(struct perf_tool *tool,
                         pr_debug("problem adding lbr entry, skipping event\n");
                         return -1;
                 }
+        } else if (rep->mem_mode == 1) {
+                if (perf_report__add_mem_hist_entry(tool, &al, sample,
+                                                    evsel, machine, event)) {
+                        pr_debug("problem adding mem entry, skipping event\n");
+                        return -1;
+                }
         } else {
                 if (al.map != NULL)
                         al.map->dso->hit = 1;
@@ -303,7 +403,8 @@ static void sig_handler(int sig __maybe_unused)
         session_done = 1;
 }
 
-static size_t hists__fprintf_nr_sample_events(struct hists *self,
+static size_t hists__fprintf_nr_sample_events(struct perf_report *rep,
+                                              struct hists *self,
                                               const char *evname, FILE *fp)
 {
         size_t ret;
@@ -314,7 +415,7 @@ static size_t hists__fprintf_nr_sample_events(struct hists *self,
         char buf[512];
         size_t size = sizeof(buf);
 
-        if (symbol_conf.event_group && evsel->nr_members > 1) {
+        if (perf_evsel__is_group_event(evsel)) {
                 struct perf_evsel *pos;
 
                 perf_evsel__group_desc(evsel, buf, size);
@@ -331,7 +432,11 @@ static size_t hists__fprintf_nr_sample_events(struct hists *self,
         if (evname != NULL)
                 ret += fprintf(fp, " of event '%s'", evname);
 
-        ret += fprintf(fp, "\n# Event count (approx.): %" PRIu64, nr_events);
+        if (rep->mem_mode) {
+                ret += fprintf(fp, "\n# Total weight : %" PRIu64, nr_events);
+                ret += fprintf(fp, "\n# Sort order   : %s", sort_order);
+        } else
+                ret += fprintf(fp, "\n# Event count (approx.): %" PRIu64, nr_events);
         return ret + fprintf(fp, "\n#\n");
 }
 
@@ -349,7 +454,7 @@ static int perf_evlist__tty_browse_hists(struct perf_evlist *evlist,
                     !perf_evsel__is_group_leader(pos))
                         continue;
 
-                hists__fprintf_nr_sample_events(hists, evname, stdout);
+                hists__fprintf_nr_sample_events(rep, hists, evname, stdout);
                 hists__fprintf(hists, true, 0, 0, stdout);
                 fprintf(stdout, "\n\n");
         }
@@ -645,7 +750,9 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
                     "Use the stdio interface"),
         OPT_STRING('s', "sort", &sort_order, "key[,key2...]",
                    "sort by key(s): pid, comm, dso, symbol, parent, cpu, srcline,"
-                   " dso_to, dso_from, symbol_to, symbol_from, mispredict"),
+                   " dso_to, dso_from, symbol_to, symbol_from, mispredict,"
+                   " weight, local_weight, mem, symbol_daddr, dso_daddr, tlb, "
+                   "snoop, locked"),
         OPT_BOOLEAN(0, "showcpuutilization", &symbol_conf.show_cpu_utilization,
                     "Show sample percentage for different cpu modes"),
         OPT_STRING('p', "parent", &parent_pattern, "regex",
@@ -693,6 +800,9 @@ int cmd_report(int argc, const char **argv, const char *prefix __maybe_unused)
                     "use branch records for histogram filling", parse_branch_mode),
         OPT_STRING(0, "objdump", &objdump_path, "path",
                    "objdump binary to use for disassembly and annotations"),
+        OPT_BOOLEAN(0, "demangle", &symbol_conf.demangle,
+                    "Disable symbol demangling"),
+        OPT_BOOLEAN(0, "mem-mode", &report.mem_mode, "mem access profile"),
         OPT_END()
         };
 
@@ -750,12 +860,24 @@ repeat:
                                      "dso_to,symbol_to";
 
         }
+        if (report.mem_mode) {
+                if (sort__branch_mode == 1) {
+                        fprintf(stderr, "branch and mem mode incompatible\n");
+                        goto error;
+                }
+                /*
+                 * if no sort_order is provided, then specify
+                 * branch-mode specific order
+                 */
+                if (sort_order == default_sort_order)
+                        sort_order = "local_weight,mem,sym,dso,symbol_daddr,dso_daddr,snoop,tlb,locked";
+        }
 
         if (setup_sorting() < 0)
                 usage_with_options(report_usage, options);
 
         /*
-         * Only in the newt browser we are doing integrated annotation,
+         * Only in the TUI browser we are doing integrated annotation,
          * so don't allocate extra space that won't be used in the stdio
          * implementation.
          */
@@ -815,6 +937,14 @@ repeat:
                 sort_entry__setup_elide(&sort_sym_from, symbol_conf.sym_from_list, "sym_from", stdout);
                 sort_entry__setup_elide(&sort_sym_to, symbol_conf.sym_to_list, "sym_to", stdout);
         } else {
+                if (report.mem_mode) {
+                        sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "symbol_daddr", stdout);
+                        sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "dso_daddr", stdout);
+                        sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "mem", stdout);
+                        sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "local_weight", stdout);
+                        sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "tlb", stdout);
+                        sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "snoop", stdout);
+                }
                 sort_entry__setup_elide(&sort_dso, symbol_conf.dso_list, "dso", stdout);
                 sort_entry__setup_elide(&sort_sym, symbol_conf.sym_list, "symbol", stdout);
         }
diff --git a/tools/perf/builtin-sched.c b/tools/perf/builtin-sched.c
index 138229439a93..2da2a6ca22bf 100644
--- a/tools/perf/builtin-sched.c
+++ b/tools/perf/builtin-sched.c
@@ -1671,7 +1671,6 @@ int cmd_sched(int argc, const char **argv, const char *prefix __maybe_unused)
                         .sample          = perf_sched__process_tracepoint_sample,
                         .comm            = perf_event__process_comm,
                         .lost            = perf_event__process_lost,
-                        .exit            = perf_event__process_exit,
                         .fork            = perf_event__process_fork,
                         .ordered_samples = true,
                 },
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index 99848761f573..7e910bab1097 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -68,7 +68,7 @@
 static void print_stat(int argc, const char **argv);
 static void print_counter_aggr(struct perf_evsel *counter, char *prefix);
 static void print_counter(struct perf_evsel *counter, char *prefix);
-static void print_aggr_socket(char *prefix);
+static void print_aggr(char *prefix);
 
 static struct perf_evlist       *evsel_list;
 
@@ -76,11 +76,17 @@ static struct perf_target	target = {
         .uid    = UINT_MAX,
 };
 
+enum aggr_mode {
+        AGGR_NONE,
+        AGGR_GLOBAL,
+        AGGR_SOCKET,
+        AGGR_CORE,
+};
+
 static int                      run_count                       =  1;
 static bool                     no_inherit                      = false;
 static bool                     scale                           =  true;
-static bool                     no_aggr                         = false;
-static bool                     aggr_socket                     = false;
+static enum aggr_mode           aggr_mode                       = AGGR_GLOBAL;
 static pid_t                    child_pid                       = -1;
 static bool                     null_run                        =  false;
 static int                      detailed_run                    =  0;
@@ -94,8 +100,10 @@ static const char		*pre_cmd			= NULL;
 static const char               *post_cmd                       = NULL;
 static bool                     sync_run                        = false;
 static unsigned int             interval                        = 0;
+static bool                     forever                         = false;
 static struct timespec          ref_time;
-static struct cpu_map           *sock_map;
+static struct cpu_map           *aggr_map;
+static int                      (*aggr_get_id)(struct cpu_map *m, int cpu);
 
 static volatile int done = 0;
 
@@ -125,6 +133,11 @@ static inline int perf_evsel__nr_cpus(struct perf_evsel *evsel)
         return perf_evsel__cpus(evsel)->nr;
 }
 
+static void perf_evsel__reset_stat_priv(struct perf_evsel *evsel)
+{
+        memset(evsel->priv, 0, sizeof(struct perf_stat));
+}
+
 static int perf_evsel__alloc_stat_priv(struct perf_evsel *evsel)
 {
         evsel->priv = zalloc(sizeof(struct perf_stat));
@@ -160,6 +173,35 @@ static void perf_evsel__free_prev_raw_counts(struct perf_evsel *evsel)
         evsel->prev_raw_counts = NULL;
 }
 
+static void perf_evlist__free_stats(struct perf_evlist *evlist)
+{
+        struct perf_evsel *evsel;
+
+        list_for_each_entry(evsel, &evlist->entries, node) {
+                perf_evsel__free_stat_priv(evsel);
+                perf_evsel__free_counts(evsel);
+                perf_evsel__free_prev_raw_counts(evsel);
+        }
+}
+
+static int perf_evlist__alloc_stats(struct perf_evlist *evlist, bool alloc_raw)
+{
+        struct perf_evsel *evsel;
+
+        list_for_each_entry(evsel, &evlist->entries, node) {
+                if (perf_evsel__alloc_stat_priv(evsel) < 0 ||
+                    perf_evsel__alloc_counts(evsel, perf_evsel__nr_cpus(evsel)) < 0 ||
+                    (alloc_raw && perf_evsel__alloc_prev_raw_counts(evsel) < 0))
+                        goto out_free;
+        }
+
+        return 0;
+
+out_free:
+        perf_evlist__free_stats(evlist);
+        return -1;
+}
+
 static struct stats runtime_nsecs_stats[MAX_NR_CPUS];
 static struct stats runtime_cycles_stats[MAX_NR_CPUS];
 static struct stats runtime_stalled_cycles_front_stats[MAX_NR_CPUS];
@@ -173,6 +215,29 @@ static struct stats runtime_itlb_cache_stats[MAX_NR_CPUS];
 static struct stats runtime_dtlb_cache_stats[MAX_NR_CPUS];
 static struct stats walltime_nsecs_stats;
 
+static void perf_stat__reset_stats(struct perf_evlist *evlist)
+{
+        struct perf_evsel *evsel;
+
+        list_for_each_entry(evsel, &evlist->entries, node) {
+                perf_evsel__reset_stat_priv(evsel);
+                perf_evsel__reset_counts(evsel, perf_evsel__nr_cpus(evsel));
+        }
+
+        memset(runtime_nsecs_stats, 0, sizeof(runtime_nsecs_stats));
+        memset(runtime_cycles_stats, 0, sizeof(runtime_cycles_stats));
+        memset(runtime_stalled_cycles_front_stats, 0, sizeof(runtime_stalled_cycles_front_stats));
+        memset(runtime_stalled_cycles_back_stats, 0, sizeof(runtime_stalled_cycles_back_stats));
+        memset(runtime_branches_stats, 0, sizeof(runtime_branches_stats));
+        memset(runtime_cacherefs_stats, 0, sizeof(runtime_cacherefs_stats));
+        memset(runtime_l1_dcache_stats, 0, sizeof(runtime_l1_dcache_stats));
+        memset(runtime_l1_icache_stats, 0, sizeof(runtime_l1_icache_stats));
+        memset(runtime_ll_cache_stats, 0, sizeof(runtime_ll_cache_stats));
+        memset(runtime_itlb_cache_stats, 0, sizeof(runtime_itlb_cache_stats));
+        memset(runtime_dtlb_cache_stats, 0, sizeof(runtime_dtlb_cache_stats));
+        memset(&walltime_nsecs_stats, 0, sizeof(walltime_nsecs_stats));
+}
+
 static int create_perf_stat_counter(struct perf_evsel *evsel)
 {
         struct perf_event_attr *attr = &evsel->attr;
@@ -249,7 +314,7 @@ static int read_counter_aggr(struct perf_evsel *counter)
         int i;
 
         if (__perf_evsel__read(counter, perf_evsel__nr_cpus(counter),
-                               evsel_list->threads->nr, scale) < 0)
+                               thread_map__nr(evsel_list->threads), scale) < 0)
                 return -1;
 
         for (i = 0; i < 3; i++)
@@ -297,56 +362,68 @@ static void print_interval(void)
         struct timespec ts, rs;
         char prefix[64];
 
-        if (no_aggr) {
+        if (aggr_mode == AGGR_GLOBAL) {
                 list_for_each_entry(counter, &evsel_list->entries, node) {
                         ps = counter->priv;
                         memset(ps->res_stats, 0, sizeof(ps->res_stats));
-                        read_counter(counter);
+                        read_counter_aggr(counter);
                 }
-        } else {
+        } else  {
                 list_for_each_entry(counter, &evsel_list->entries, node) {
                         ps = counter->priv;
                         memset(ps->res_stats, 0, sizeof(ps->res_stats));
-                        read_counter_aggr(counter);
+                        read_counter(counter);
                 }
         }
+
         clock_gettime(CLOCK_MONOTONIC, &ts);
         diff_timespec(&rs, &ts, &ref_time);
         sprintf(prefix, "%6lu.%09lu%s", rs.tv_sec, rs.tv_nsec, csv_sep);
 
         if (num_print_interval == 0 && !csv_output) {
-                if (aggr_socket)
+                switch (aggr_mode) {
+                case AGGR_SOCKET:
                         fprintf(output, "#           time socket cpus             counts events\n");
-                else if (no_aggr)
+                        break;
+                case AGGR_CORE:
+                        fprintf(output, "#           time core         cpus             counts events\n");
+                        break;
+                case AGGR_NONE:
                         fprintf(output, "#           time CPU                 counts events\n");
-                else
+                        break;
+                case AGGR_GLOBAL:
+                default:
                         fprintf(output, "#           time             counts events\n");
+                }
         }
 
         if (++num_print_interval == 25)
                 num_print_interval = 0;
 
-        if (aggr_socket)
-                print_aggr_socket(prefix);
-        else if (no_aggr) {
+        switch (aggr_mode) {
+        case AGGR_CORE:
+        case AGGR_SOCKET:
+                print_aggr(prefix);
+                break;
+        case AGGR_NONE:
                 list_for_each_entry(counter, &evsel_list->entries, node)
                         print_counter(counter, prefix);
-        } else {
+                break;
+        case AGGR_GLOBAL:
+        default:
                 list_for_each_entry(counter, &evsel_list->entries, node)
                         print_counter_aggr(counter, prefix);
         }
 }
 
-static int __run_perf_stat(int argc __maybe_unused, const char **argv)
+static int __run_perf_stat(int argc, const char **argv)
 {
         char msg[512];
         unsigned long long t0, t1;
         struct perf_evsel *counter;
         struct timespec ts;
         int status = 0;
-        int child_ready_pipe[2], go_pipe[2];
         const bool forks = (argc > 0);
-        char buf;
 
         if (interval) {
                 ts.tv_sec  = interval / 1000;
@@ -356,61 +433,12 @@ static int __run_perf_stat(int argc __maybe_unused, const char **argv)
                 ts.tv_nsec = 0;
         }
 
-        if (aggr_socket
-            && cpu_map__build_socket_map(evsel_list->cpus, &sock_map)) {
-                perror("cannot build socket map");
-                return -1;
-        }
-
-        if (forks && (pipe(child_ready_pipe) < 0 || pipe(go_pipe) < 0)) {
-                perror("failed to create pipes");
-                return -1;
-        }
-
         if (forks) {
-                if ((child_pid = fork()) < 0)
-                        perror("failed to fork");
-
-                if (!child_pid) {
-                        close(child_ready_pipe[0]);
-                        close(go_pipe[1]);
-                        fcntl(go_pipe[0], F_SETFD, FD_CLOEXEC);
-
-                        /*
-                         * Do a dummy execvp to get the PLT entry resolved,
-                         * so we avoid the resolver overhead on the real
-                         * execvp call.
-                         */
-                        execvp("", (char **)argv);
-
-                        /*
-                         * Tell the parent we're ready to go
-                         */
-                        close(child_ready_pipe[1]);
-
-                        /*
-                         * Wait until the parent tells us to go.
-                         */
-                        if (read(go_pipe[0], &buf, 1) == -1)
-                                perror("unable to read pipe");
-
-                        execvp(argv[0], (char **)argv);
-
-                        perror(argv[0]);
-                        exit(-1);
+                if (perf_evlist__prepare_workload(evsel_list, &target, argv,
+                                                  false, false) < 0) {
+                        perror("failed to prepare workload");
+                        return -1;
                 }
-
-                if (perf_target__none(&target))
-                        evsel_list->threads->map[0] = child_pid;
-
-                /*
-                 * Wait for the child to be ready to exec.
-                 */
-                close(child_ready_pipe[1]);
-                close(go_pipe[0]);
-                if (read(child_ready_pipe[0], &buf, 1) == -1)
-                        perror("unable to read pipe");
-                close(child_ready_pipe[0]);
         }
 
         if (group)
@@ -457,7 +485,8 @@ static int __run_perf_stat(int argc __maybe_unused, const char **argv)
         clock_gettime(CLOCK_MONOTONIC, &ref_time);
 
         if (forks) {
-                close(go_pipe[1]);
+                perf_evlist__start_workload(evsel_list);
+
                 if (interval) {
                         while (!waitpid(child_pid, &status, WNOHANG)) {
                                 nanosleep(&ts, NULL);
@@ -479,16 +508,16 @@ static int __run_perf_stat(int argc __maybe_unused, const char **argv)
 
         update_stats(&walltime_nsecs_stats, t1 - t0);
 
-        if (no_aggr) {
+        if (aggr_mode == AGGR_GLOBAL) {
                 list_for_each_entry(counter, &evsel_list->entries, node) {
-                        read_counter(counter);
-                        perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter), 1);
+                        read_counter_aggr(counter);
+                        perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter),
+                                             thread_map__nr(evsel_list->threads));
                 }
         } else {
                 list_for_each_entry(counter, &evsel_list->entries, node) {
-                        read_counter_aggr(counter);
-                        perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter),
-                                             evsel_list->threads->nr);
+                        read_counter(counter);
+                        perf_evsel__close_fd(counter, perf_evsel__nr_cpus(counter), 1);
                 }
         }
 
@@ -542,26 +571,47 @@ static void print_noise(struct perf_evsel *evsel, double avg)
         print_noise_pct(stddev_stats(&ps->res_stats[0]), avg);
 }
 
-static void nsec_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
+static void aggr_printout(struct perf_evsel *evsel, int id, int nr)
 {
-        double msecs = avg / 1e6;
-        char cpustr[16] = { '\0', };
-        const char *fmt = csv_output ? "%s%.6f%s%s" : "%s%18.6f%s%-25s";
-
-        if (aggr_socket)
-                sprintf(cpustr, "S%*d%s%*d%s",
+        switch (aggr_mode) {
+        case AGGR_CORE:
+                fprintf(output, "S%d-C%*d%s%*d%s",
+                        cpu_map__id_to_socket(id),
+                        csv_output ? 0 : -8,
+                        cpu_map__id_to_cpu(id),
+                        csv_sep,
+                        csv_output ? 0 : 4,
+                        nr,
+                        csv_sep);
+                break;
+        case AGGR_SOCKET:
+                fprintf(output, "S%*d%s%*d%s",
                         csv_output ? 0 : -5,
-                        cpu,
+                        id,
                         csv_sep,
                         csv_output ? 0 : 4,
                         nr,
                         csv_sep);
-        else if (no_aggr)
-                sprintf(cpustr, "CPU%*d%s",
+                        break;
+        case AGGR_NONE:
+                fprintf(output, "CPU%*d%s",
                         csv_output ? 0 : -4,
-                        perf_evsel__cpus(evsel)->map[cpu], csv_sep);
+                        perf_evsel__cpus(evsel)->map[id], csv_sep);
+                break;
+        case AGGR_GLOBAL:
+        default:
+                break;
+        }
+}
+
+static void nsec_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
+{
+        double msecs = avg / 1e6;
+        const char *fmt = csv_output ? "%.6f%s%s" : "%18.6f%s%-25s";
+
+        aggr_printout(evsel, cpu, nr);
 
-        fprintf(output, fmt, cpustr, msecs, csv_sep, perf_evsel__name(evsel));
+        fprintf(output, fmt, msecs, csv_sep, perf_evsel__name(evsel));
 
         if (evsel->cgrp)
                 fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
@@ -758,32 +808,21 @@ static void print_ll_cache_misses(int cpu,
 static void abs_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
 {
         double total, ratio = 0.0;
-        char cpustr[16] = { '\0', };
         const char *fmt;
 
         if (csv_output)
-                fmt = "%s%.0f%s%s";
+                fmt = "%.0f%s%s";
         else if (big_num)
-                fmt = "%s%'18.0f%s%-25s";
+                fmt = "%'18.0f%s%-25s";
         else
-                fmt = "%s%18.0f%s%-25s";
+                fmt = "%18.0f%s%-25s";
 
-        if (aggr_socket)
-                sprintf(cpustr, "S%*d%s%*d%s",
-                        csv_output ? 0 : -5,
-                        cpu,
-                        csv_sep,
-                        csv_output ? 0 : 4,
-                        nr,
-                        csv_sep);
-        else if (no_aggr)
-                sprintf(cpustr, "CPU%*d%s",
-                        csv_output ? 0 : -4,
-                        perf_evsel__cpus(evsel)->map[cpu], csv_sep);
-        else
+        aggr_printout(evsel, cpu, nr);
+
+        if (aggr_mode == AGGR_GLOBAL)
                 cpu = 0;
 
-        fprintf(output, fmt, cpustr, avg, csv_sep, perf_evsel__name(evsel));
+        fprintf(output, fmt, avg, csv_sep, perf_evsel__name(evsel));
 
         if (evsel->cgrp)
                 fprintf(output, "%s%s", csv_sep, evsel->cgrp->name);
@@ -882,23 +921,23 @@ static void abs_printout(int cpu, int nr, struct perf_evsel *evsel, double avg)
         }
 }
 
-static void print_aggr_socket(char *prefix)
+static void print_aggr(char *prefix)
 {
         struct perf_evsel *counter;
+        int cpu, s, s2, id, nr;
         u64 ena, run, val;
-        int cpu, s, s2, sock, nr;
 
-        if (!sock_map)
+        if (!(aggr_map || aggr_get_id))
                 return;
 
-        for (s = 0; s < sock_map->nr; s++) {
-                sock = cpu_map__socket(sock_map, s);
+        for (s = 0; s < aggr_map->nr; s++) {
+                id = aggr_map->map[s];
                 list_for_each_entry(counter, &evsel_list->entries, node) {
                         val = ena = run = 0;
                         nr = 0;
                         for (cpu = 0; cpu < perf_evsel__nr_cpus(counter); cpu++) {
-                                s2 = cpu_map__get_socket(evsel_list->cpus, cpu);
-                                if (s2 != sock)
+                                s2 = aggr_get_id(evsel_list->cpus, cpu);
+                                if (s2 != id)
                                         continue;
                                 val += counter->counts->cpu[cpu].val;
                                 ena += counter->counts->cpu[cpu].ena;
@@ -909,18 +948,15 @@ static void print_aggr_socket(char *prefix)
                                 fprintf(output, "%s", prefix);
 
                         if (run == 0 || ena == 0) {
-                                fprintf(output, "S%*d%s%*d%s%*s%s%*s",
-                                        csv_output ? 0 : -5,
-                                        s,
-                                        csv_sep,
-                                        csv_output ? 0 : 4,
-                                        nr,
-                                        csv_sep,
+                                aggr_printout(counter, cpu, nr);
+
+                                fprintf(output, "%*s%s%*s",
                                         csv_output ? 0 : 18,
                                         counter->supported ? CNTR_NOT_COUNTED : CNTR_NOT_SUPPORTED,
                                         csv_sep,
                                         csv_output ? 0 : -24,
                                         perf_evsel__name(counter));
+
                                 if (counter->cgrp)
                                         fprintf(output, "%s%s",
                                                 csv_sep, counter->cgrp->name);
@@ -930,9 +966,9 @@ static void print_aggr_socket(char *prefix)
                         }
 
                         if (nsec_counter(counter))
-                                nsec_printout(sock, nr, counter, val);
+                                nsec_printout(id, nr, counter, val);
                         else
-                                abs_printout(sock, nr, counter, val);
+                                abs_printout(id, nr, counter, val);
 
                         if (!csv_output) {
                                 print_noise(counter, 1.0);
@@ -1073,14 +1109,21 @@ static void print_stat(int argc, const char **argv)
                 fprintf(output, ":\n\n");
         }
 
-        if (aggr_socket)
-                print_aggr_socket(NULL);
-        else if (no_aggr) {
-                list_for_each_entry(counter, &evsel_list->entries, node)
-                        print_counter(counter, NULL);
-        } else {
+        switch (aggr_mode) {
+        case AGGR_CORE:
+        case AGGR_SOCKET:
+                print_aggr(NULL);
+                break;
+        case AGGR_GLOBAL:
                 list_for_each_entry(counter, &evsel_list->entries, node)
                         print_counter_aggr(counter, NULL);
+                break;
+        case AGGR_NONE:
+                list_for_each_entry(counter, &evsel_list->entries, node)
+                        print_counter(counter, NULL);
+                break;
+        default:
+                break;
         }
 
         if (!csv_output) {
@@ -1126,6 +1169,32 @@ static int stat__set_big_num(const struct option *opt __maybe_unused,
         return 0;
 }
 
+static int perf_stat_init_aggr_mode(void)
+{
+        switch (aggr_mode) {
+        case AGGR_SOCKET:
+                if (cpu_map__build_socket_map(evsel_list->cpus, &aggr_map)) {
+                        perror("cannot build socket map");
+                        return -1;
+                }
+                aggr_get_id = cpu_map__get_socket;
+                break;
+        case AGGR_CORE:
+                if (cpu_map__build_core_map(evsel_list->cpus, &aggr_map)) {
+                        perror("cannot build core map");
+                        return -1;
+                }
+                aggr_get_id = cpu_map__get_core;
+                break;
+        case AGGR_NONE:
+        case AGGR_GLOBAL:
+        default:
+                break;
+        }
+        return 0;
+}
+
+
 /*
  * Add default attributes, if there were no attributes specified or
  * if -d/--detailed, -d -d or -d -d -d is used:
@@ -1296,7 +1365,7 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
         OPT_INCR('v', "verbose", &verbose,
                     "be more verbose (show counter open errors, etc)"),
         OPT_INTEGER('r', "repeat", &run_count,
-                    "repeat command and print average + stddev (max: 100)"),
+                    "repeat command and print average + stddev (max: 100, forever: 0)"),
         OPT_BOOLEAN('n', "null", &null_run,
                     "null run - dont start any counters"),
         OPT_INCR('d', "detailed", &detailed_run,
@@ -1308,7 +1377,8 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
                            stat__set_big_num),
         OPT_STRING('C', "cpu", &target.cpu_list, "cpu",
                     "list of cpus to monitor in system-wide"),
-        OPT_BOOLEAN('A', "no-aggr", &no_aggr, "disable CPU count aggregation"),
+        OPT_SET_UINT('A', "no-aggr", &aggr_mode,
+                    "disable CPU count aggregation", AGGR_NONE),
         OPT_STRING('x', "field-separator", &csv_sep, "separator",
                    "print counts with custom separator"),
         OPT_CALLBACK('G', "cgroup", &evsel_list, "name",
@@ -1323,20 +1393,22 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
                         "command to run after to the measured command"),
         OPT_UINTEGER('I', "interval-print", &interval,
                     "print counts at regular interval in ms (>= 100)"),
-        OPT_BOOLEAN(0, "aggr-socket", &aggr_socket, "aggregate counts per processor socket"),
+        OPT_SET_UINT(0, "per-socket", &aggr_mode,
+                     "aggregate counts per processor socket", AGGR_SOCKET),
+        OPT_SET_UINT(0, "per-core", &aggr_mode,
+                     "aggregate counts per physical processor core", AGGR_CORE),
         OPT_END()
         };
         const char * const stat_usage[] = {
                 "perf stat [<options>] [<command>]",
                 NULL
         };
-        struct perf_evsel *pos;
         int status = -ENOMEM, run_idx;
         const char *mode;
 
         setlocale(LC_ALL, "");
 
-        evsel_list = perf_evlist__new(NULL, NULL);
+        evsel_list = perf_evlist__new();
         if (evsel_list == NULL)
                 return -ENOMEM;
 
@@ -1399,23 +1471,21 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
 
         if (!argc && !perf_target__has_task(&target))
                 usage_with_options(stat_usage, options);
-        if (run_count <= 0)
+        if (run_count < 0) {
                 usage_with_options(stat_usage, options);
+        } else if (run_count == 0) {
+                forever = true;
+                run_count = 1;
+        }
 
         /* no_aggr, cgroup are for system-wide only */
-        if ((no_aggr || nr_cgroups) && !perf_target__has_cpu(&target)) {
+        if ((aggr_mode != AGGR_GLOBAL || nr_cgroups)
+             && !perf_target__has_cpu(&target)) {
                 fprintf(stderr, "both cgroup and no-aggregation "
                         "modes only available in system-wide mode\n");
 
                 usage_with_options(stat_usage, options);
-        }
-
-        if (aggr_socket) {
-                if (!perf_target__has_cpu(&target)) {
-                        fprintf(stderr, "--aggr-socket only available in system-wide mode (-a)\n");
-                        usage_with_options(stat_usage, options);
-                }
-                no_aggr = true;
+                return -1;
         }
 
         if (add_default_attributes())
@@ -1438,17 +1508,11 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
                 return -1;
         }
 
-        list_for_each_entry(pos, &evsel_list->entries, node) {
-                if (perf_evsel__alloc_stat_priv(pos) < 0 ||
-                    perf_evsel__alloc_counts(pos, perf_evsel__nr_cpus(pos)) < 0)
-                        goto out_free_fd;
-        }
-        if (interval) {
-                list_for_each_entry(pos, &evsel_list->entries, node) {
-                        if (perf_evsel__alloc_prev_raw_counts(pos) < 0)
-                                goto out_free_fd;
-                }
-        }
+        if (perf_evlist__alloc_stats(evsel_list, interval))
+                goto out_free_maps;
+
+        if (perf_stat_init_aggr_mode())
+                goto out;
 
         /*
          * We dont want to block the signals - that would cause
@@ -1457,28 +1521,30 @@ int cmd_stat(int argc, const char **argv, const char *prefix __maybe_unused)
          * task, but being ignored by perf stat itself:
          */
         atexit(sig_atexit);
-        signal(SIGINT,  skip_signal);
+        if (!forever)
+                signal(SIGINT,  skip_signal);
         signal(SIGCHLD, skip_signal);
         signal(SIGALRM, skip_signal);
         signal(SIGABRT, skip_signal);
 
         status = 0;
-        for (run_idx = 0; run_idx < run_count; run_idx++) {
+        for (run_idx = 0; forever || run_idx < run_count; run_idx++) {
                 if (run_count != 1 && verbose)
                         fprintf(output, "[ perf stat: executing run #%d ... ]\n",
                                 run_idx + 1);
 
                 status = run_perf_stat(argc, argv);
+                if (forever && status != -1) {
+                        print_stat(argc, argv);
+                        perf_stat__reset_stats(evsel_list);
+                }
         }
 
-        if (status != -1 && !interval)
+        if (!forever && status != -1 && !interval)
                 print_stat(argc, argv);
-out_free_fd:
-        list_for_each_entry(pos, &evsel_list->entries, node) {
-                perf_evsel__free_stat_priv(pos);
-                perf_evsel__free_counts(pos);
-                perf_evsel__free_prev_raw_counts(pos);
-        }
+
+        perf_evlist__free_stats(evsel_list);
+out_free_maps:
         perf_evlist__delete_maps(evsel_list);
 out:
         perf_evlist__delete(evsel_list);
diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 72f6eb7b4173..67bdb9f14ad6 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -231,7 +231,7 @@ static void perf_top__show_details(struct perf_top *top)
         printf("Showing %s for %s\n", perf_evsel__name(top->sym_evsel), symbol->name);
         printf("  Events  Pcnt (>=%d%%)\n", top->sym_pcnt_filter);
 
-        more = symbol__annotate_printf(symbol, he->ms.map, top->sym_evsel->idx,
+        more = symbol__annotate_printf(symbol, he->ms.map, top->sym_evsel,
                                        0, top->sym_pcnt_filter, top->print_entries, 4);
         if (top->zero)
                 symbol__annotate_zero_histogram(symbol, top->sym_evsel->idx);
@@ -251,7 +251,8 @@ static struct hist_entry *perf_evsel__add_hist_entry(struct perf_evsel *evsel,
 {
         struct hist_entry *he;
 
-        he = __hists__add_entry(&evsel->hists, al, NULL, sample->period);
+        he = __hists__add_entry(&evsel->hists, al, NULL, sample->period,
+                                sample->weight);
         if (he == NULL)
                 return NULL;
 
@@ -1088,7 +1089,7 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)
         OPT_INCR('v', "verbose", &verbose,
                     "be more verbose (show counter open errors, etc)"),
         OPT_STRING('s', "sort", &sort_order, "key[,key2...]",
-                   "sort by key(s): pid, comm, dso, symbol, parent"),
+                   "sort by key(s): pid, comm, dso, symbol, parent, weight, local_weight"),
         OPT_BOOLEAN('n', "show-nr-samples", &symbol_conf.show_nr_samples,
                     "Show a column with the number of samples"),
         OPT_CALLBACK_DEFAULT('G', "call-graph", &top.record_opts,
@@ -1116,7 +1117,7 @@ int cmd_top(int argc, const char **argv, const char *prefix __maybe_unused)
                 NULL
         };
 
-        top.evlist = perf_evlist__new(NULL, NULL);
+        top.evlist = perf_evlist__new();
         if (top.evlist == NULL)
                 return -ENOMEM;
 
diff --git a/tools/perf/builtin-trace.c b/tools/perf/builtin-trace.c
index d222d7fc7e96..ab3ed4af1466 100644
--- a/tools/perf/builtin-trace.c
+++ b/tools/perf/builtin-trace.c
@@ -419,7 +419,7 @@ out_dump:
 
 static int trace__run(struct trace *trace, int argc, const char **argv)
 {
-        struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+        struct perf_evlist *evlist = perf_evlist__new();
         struct perf_evsel *evsel;
         int err = -1, i;
         unsigned long before;
@@ -452,7 +452,7 @@ static int trace__run(struct trace *trace, int argc, const char **argv)
         err = trace__symbols_init(trace, evlist);
         if (err < 0) {
                 printf("Problems initializing symbol libraries!\n");
-                goto out_delete_evlist;
+                goto out_delete_maps;
         }
 
         perf_evlist__config(evlist, &trace->opts);
@@ -461,23 +461,24 @@ static int trace__run(struct trace *trace, int argc, const char **argv)
         signal(SIGINT, sig_handler);
 
         if (forks) {
-                err = perf_evlist__prepare_workload(evlist, &trace->opts, argv);
+                err = perf_evlist__prepare_workload(evlist, &trace->opts.target,
+                                                    argv, false, false);
                 if (err < 0) {
                         printf("Couldn't run the workload!\n");
-                        goto out_delete_evlist;
+                        goto out_delete_maps;
                 }
         }
 
         err = perf_evlist__open(evlist);
         if (err < 0) {
                 printf("Couldn't create the events: %s\n", strerror(errno));
-                goto out_delete_evlist;
+                goto out_delete_maps;
         }
 
         err = perf_evlist__mmap(evlist, UINT_MAX, false);
         if (err < 0) {
                 printf("Couldn't mmap the events: %s\n", strerror(errno));
-                goto out_delete_evlist;
+                goto out_close_evlist;
         }
 
         perf_evlist__enable(evlist);
@@ -526,13 +527,6 @@ again:
                                 continue;
                         }
 
-                        if (sample.raw_data == NULL) {
-                                printf("%s sample with no payload for tid: %d, cpu %d, raw_size=%d, skipping...\n",
-                                       perf_evsel__name(evsel), sample.tid,
-                                       sample.cpu, sample.raw_size);
-                                continue;
-                        }
-
                         handler = evsel->handler.func;
                         handler(trace, evsel, &sample);
                 }
@@ -540,7 +534,7 @@ again:
 
         if (trace->nr_events == before) {
                 if (done)
-                        goto out_delete_evlist;
+                        goto out_unmap_evlist;
 
                 poll(evlist->pollfd, evlist->nr_fds, -1);
         }
@@ -550,6 +544,12 @@ again:
 
         goto again;
 
+out_unmap_evlist:
+        perf_evlist__munmap(evlist);
+out_close_evlist:
+        perf_evlist__close(evlist);
+out_delete_maps:
+        perf_evlist__delete_maps(evlist);
 out_delete_evlist:
         perf_evlist__delete(evlist);
 out:
diff --git a/tools/perf/builtin.h b/tools/perf/builtin.h
index 08143bd854c7..b210d62907e4 100644
--- a/tools/perf/builtin.h
+++ b/tools/perf/builtin.h
@@ -36,6 +36,7 @@ extern int cmd_kvm(int argc, const char **argv, const char *prefix);
 extern int cmd_test(int argc, const char **argv, const char *prefix);
 extern int cmd_trace(int argc, const char **argv, const char *prefix);
 extern int cmd_inject(int argc, const char **argv, const char *prefix);
+extern int cmd_mem(int argc, const char **argv, const char *prefix);
 
 extern int find_scripts(char **scripts_array, char **scripts_path_array);
 #endif
diff --git a/tools/perf/command-list.txt b/tools/perf/command-list.txt
index 3e86bbd8c2d5..0906fc401c52 100644
--- a/tools/perf/command-list.txt
+++ b/tools/perf/command-list.txt
@@ -10,17 +10,18 @@ perf-buildid-list		mainporcelain common
 perf-diff                       mainporcelain common
 perf-evlist                     mainporcelain common
 perf-inject                     mainporcelain common
+perf-kmem                       mainporcelain common
+perf-kvm                        mainporcelain common
 perf-list                       mainporcelain common
-perf-sched                      mainporcelain common
+perf-lock                       mainporcelain common
+perf-mem                        mainporcelain common
+perf-probe                      mainporcelain full
 perf-record                     mainporcelain common
 perf-report                     mainporcelain common
+perf-sched                      mainporcelain common
+perf-script                     mainporcelain common
 perf-stat                       mainporcelain common
+perf-test                       mainporcelain common
 perf-timechart                  mainporcelain common
 perf-top                        mainporcelain common
 perf-trace                      mainporcelain common
-perf-script                     mainporcelain common
-perf-probe                      mainporcelain full
-perf-kmem                       mainporcelain common
-perf-lock                       mainporcelain common
-perf-kvm                        mainporcelain common
-perf-test                       mainporcelain common
diff --git a/tools/perf/config/feature-tests.mak b/tools/perf/config/feature-tests.mak
index b4eabb44e381..708fb8e9822a 100644
--- a/tools/perf/config/feature-tests.mak
+++ b/tools/perf/config/feature-tests.mak
@@ -61,15 +61,13 @@ int main(void)
 }
 endef
 
-ifndef NO_NEWT
-define SOURCE_NEWT
-#include <newt.h>
+ifndef NO_SLANG
+define SOURCE_SLANG
+#include <slang.h>
 
 int main(void)
 {
-        newtInit();
-        newtCls();
-        return newtFinished();
+        return SLsmg_init_smg();
 }
 endef
 endif
@@ -235,4 +233,4 @@ int main(void)
         numa_available();
         return 0;
 }
-endef
\ No newline at end of file
+endef
diff --git a/tools/perf/perf.c b/tools/perf/perf.c
index 095b88207cd3..85e1aed95204 100644
--- a/tools/perf/perf.c
+++ b/tools/perf/perf.c
@@ -13,7 +13,7 @@
 #include "util/quote.h"
 #include "util/run-command.h"
 #include "util/parse-events.h"
-#include "util/debugfs.h"
+#include <lk/debugfs.h>
 #include <pthread.h>
 
 const char perf_usage_string[] =
@@ -60,6 +60,7 @@ static struct cmd_struct commands[] = {
         { "trace",      cmd_trace,      0 },
 #endif
         { "inject",     cmd_inject,     0 },
+        { "mem",        cmd_mem,        0 },
 };
 
 struct pager_config {
@@ -193,13 +194,13 @@ static int handle_options(const char ***argv, int *argc, int *envchanged)
                                 fprintf(stderr, "No directory given for --debugfs-dir.\n");
                                 usage(perf_usage_string);
                         }
-                        debugfs_set_path((*argv)[1]);
+                        perf_debugfs_set_path((*argv)[1]);
                         if (envchanged)
                                 *envchanged = 1;
                         (*argv)++;
                         (*argc)--;
                 } else if (!prefixcmp(cmd, CMD_DEBUGFS_DIR)) {
-                        debugfs_set_path(cmd + strlen(CMD_DEBUGFS_DIR));
+                        perf_debugfs_set_path(cmd + strlen(CMD_DEBUGFS_DIR));
                         fprintf(stderr, "dir: %s\n", debugfs_mountpoint);
                         if (envchanged)
                                 *envchanged = 1;
@@ -461,7 +462,7 @@ int main(int argc, const char **argv)
         if (!cmd)
                 cmd = "perf-help";
         /* get debugfs mount point from /proc/mounts */
-        debugfs_mount(NULL);
+        perf_debugfs_mount(NULL);
         /*
          * "perf-xxxx" is the same as "perf xxxx", but we obviously:
          *
@@ -517,9 +518,8 @@ int main(int argc, const char **argv)
 
         while (1) {
                 static int done_help;
-                static int was_alias;
+                int was_alias = run_argv(&argc, &argv);
 
-                was_alias = run_argv(&argc, &argv);
                 if (errno != ENOENT)
                         break;
 
diff --git a/tools/perf/perf.h b/tools/perf/perf.h
index 74659ecf93e0..32bd102c32b6 100644
--- a/tools/perf/perf.h
+++ b/tools/perf/perf.h
@@ -218,6 +218,7 @@ struct perf_record_opts {
         bool         pipe_output;
         bool         raw_samples;
         bool         sample_address;
+        bool         sample_weight;
         bool         sample_time;
         bool         period;
         unsigned int freq;
diff --git a/tools/perf/tests/attr.c b/tools/perf/tests/attr.c
index bdcceb886f77..038de3ecb8cb 100644
--- a/tools/perf/tests/attr.c
+++ b/tools/perf/tests/attr.c
@@ -147,10 +147,15 @@ void test_attr__open(struct perf_event_attr *attr, pid_t pid, int cpu,
 
 static int run_dir(const char *d, const char *perf)
 {
+        char v[] = "-vvvvv";
+        int vcnt = min(verbose, (int) sizeof(v) - 1);
         char cmd[3*PATH_MAX];
 
-        snprintf(cmd, 3*PATH_MAX, PYTHON " %s/attr.py -d %s/attr/ -p %s %s",
-                 d, d, perf, verbose ? "-v" : "");
+        if (verbose)
+                vcnt++;
+
+        snprintf(cmd, 3*PATH_MAX, PYTHON " %s/attr.py -d %s/attr/ -p %s %.*s",
+                 d, d, perf, vcnt, v);
 
         return system(cmd);
 }
diff --git a/tools/perf/tests/attr.py b/tools/perf/tests/attr.py
index 2f629ca485bc..c9b4b6269b51 100644
--- a/tools/perf/tests/attr.py
+++ b/tools/perf/tests/attr.py
@@ -24,6 +24,7 @@ class Unsup(Exception):
 
 class Event(dict):
     terms = [
+        'cpu',
         'flags',
         'type',
         'size',
@@ -121,7 +122,7 @@ class Test(object):
         parser = ConfigParser.SafeConfigParser()
         parser.read(path)
 
-        log.debug("running '%s'" % path)
+        log.warning("running '%s'" % path)
 
         self.path     = path
         self.test_dir = options.test_dir
@@ -172,7 +173,7 @@ class Test(object):
               self.perf, self.command, tempdir, self.args)
         ret = os.WEXITSTATUS(os.system(cmd))
 
-        log.warning("  running '%s' ret %d " % (cmd, ret))
+        log.info("  '%s' ret %d " % (cmd, ret))
 
         if ret != int(self.ret):
             raise Unsup(self)
diff --git a/tools/perf/tests/attr/base-record b/tools/perf/tests/attr/base-record
index 5bc3880f7be5..b4fc835de607 100644
--- a/tools/perf/tests/attr/base-record
+++ b/tools/perf/tests/attr/base-record
@@ -2,6 +2,7 @@
 fd=1
 group_fd=-1
 flags=0
+cpu=*
 type=0|1
 size=96
 config=0
diff --git a/tools/perf/tests/attr/base-stat b/tools/perf/tests/attr/base-stat
index 4bd79a82784f..748ee949a204 100644
--- a/tools/perf/tests/attr/base-stat
+++ b/tools/perf/tests/attr/base-stat
@@ -2,6 +2,7 @@
 fd=1
 group_fd=-1
 flags=0
+cpu=*
 type=0
 size=96
 config=0
diff --git a/tools/perf/tests/attr/test-record-C0 b/tools/perf/tests/attr/test-record-C0
new file mode 100644
index 000000000000..d6a7e43f61b3
--- /dev/null
+++ b/tools/perf/tests/attr/test-record-C0
@@ -0,0 +1,13 @@
+[config]
+command = record
+args    = -C 0 kill >/dev/null 2>&1
+
+[event:base-record]
+cpu=0
+
+# no enable on exec for CPU attached
+enable_on_exec=0
+
+# PERF_SAMPLE_IP | PERF_SAMPLE_TID PERF_SAMPLE_TIME | # PERF_SAMPLE_PERIOD
+# + PERF_SAMPLE_CPU added by -C 0
+sample_type=391
diff --git a/tools/perf/tests/attr/test-stat-C0 b/tools/perf/tests/attr/test-stat-C0
new file mode 100644
index 000000000000..aa835950751f
--- /dev/null
+++ b/tools/perf/tests/attr/test-stat-C0
@@ -0,0 +1,9 @@
+[config]
+command = stat
+args    = -e cycles -C 0 kill >/dev/null 2>&1
+ret     = 1
+
+[event:base-stat]
+# events are enabled by default when attached to cpu
+disabled=0
+enable_on_exec=0
diff --git a/tools/perf/tests/bp_signal.c b/tools/perf/tests/bp_signal.c
new file mode 100644
index 000000000000..68daa289e94c
--- /dev/null
+++ b/tools/perf/tests/bp_signal.c
@@ -0,0 +1,186 @@
+/*
+ * Inspired by breakpoint overflow test done by
+ * Vince Weaver <vincent.weaver@maine.edu> for perf_event_tests
+ * (git://github.com/deater/perf_event_tests)
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <time.h>
+#include <fcntl.h>
+#include <signal.h>
+#include <sys/mman.h>
+#include <linux/compiler.h>
+#include <linux/hw_breakpoint.h>
+
+#include "tests.h"
+#include "debug.h"
+#include "perf.h"
+
+static int fd1;
+static int fd2;
+static int overflows;
+
+__attribute__ ((noinline))
+static int test_function(void)
+{
+        return time(NULL);
+}
+
+static void sig_handler(int signum __maybe_unused,
+                        siginfo_t *oh __maybe_unused,
+                        void *uc __maybe_unused)
+{
+        overflows++;
+
+        if (overflows > 10) {
+                /*
+                 * This should be executed only once during
+                 * this test, if we are here for the 10th
+                 * time, consider this the recursive issue.
+                 *
+                 * We can get out of here by disable events,
+                 * so no new SIGIO is delivered.
+                 */
+                ioctl(fd1, PERF_EVENT_IOC_DISABLE, 0);
+                ioctl(fd2, PERF_EVENT_IOC_DISABLE, 0);
+        }
+}
+
+static int bp_event(void *fn, int setup_signal)
+{
+        struct perf_event_attr pe;
+        int fd;
+
+        memset(&pe, 0, sizeof(struct perf_event_attr));
+        pe.type = PERF_TYPE_BREAKPOINT;
+        pe.size = sizeof(struct perf_event_attr);
+
+        pe.config = 0;
+        pe.bp_type = HW_BREAKPOINT_X;
+        pe.bp_addr = (unsigned long) fn;
+        pe.bp_len = sizeof(long);
+
+        pe.sample_period = 1;
+        pe.sample_type = PERF_SAMPLE_IP;
+        pe.wakeup_events = 1;
+
+        pe.disabled = 1;
+        pe.exclude_kernel = 1;
+        pe.exclude_hv = 1;
+
+        fd = sys_perf_event_open(&pe, 0, -1, -1, 0);
+        if (fd < 0) {
+                pr_debug("failed opening event %llx\n", pe.config);
+                return TEST_FAIL;
+        }
+
+        if (setup_signal) {
+                fcntl(fd, F_SETFL, O_RDWR|O_NONBLOCK|O_ASYNC);
+                fcntl(fd, F_SETSIG, SIGIO);
+                fcntl(fd, F_SETOWN, getpid());
+        }
+
+        ioctl(fd, PERF_EVENT_IOC_RESET, 0);
+
+        return fd;
+}
+
+static long long bp_count(int fd)
+{
+        long long count;
+        int ret;
+
+        ret = read(fd, &count, sizeof(long long));
+        if (ret != sizeof(long long)) {
+                pr_debug("failed to read: %d\n", ret);
+                return TEST_FAIL;
+        }
+
+        return count;
+}
+
+int test__bp_signal(void)
+{
+        struct sigaction sa;
+        long long count1, count2;
+
+        /* setup SIGIO signal handler */
+        memset(&sa, 0, sizeof(struct sigaction));
+        sa.sa_sigaction = (void *) sig_handler;
+        sa.sa_flags = SA_SIGINFO;
+
+        if (sigaction(SIGIO, &sa, NULL) < 0) {
+                pr_debug("failed setting up signal handler\n");
+                return TEST_FAIL;
+        }
+
+        /*
+         * We create following events:
+         *
+         * fd1 - breakpoint event on test_function with SIGIO
+         *       signal configured. We should get signal
+         *       notification each time the breakpoint is hit
+         *
+         * fd2 - breakpoint event on sig_handler without SIGIO
+         *       configured.
+         *
+         * Following processing should happen:
+         *   - execute test_function
+         *   - fd1 event breakpoint hit -> count1 == 1
+         *   - SIGIO is delivered       -> overflows == 1
+         *   - fd2 event breakpoint hit -> count2 == 1
+         *
+         * The test case check following error conditions:
+         * - we get stuck in signal handler because of debug
+         *   exception being triggered receursively due to
+         *   the wrong RF EFLAG management
+         *
+         * - we never trigger the sig_handler breakpoint due
+         *   to the rong RF EFLAG management
+         *
+         */
+
+        fd1 = bp_event(test_function, 1);
+        fd2 = bp_event(sig_handler, 0);
+
+        ioctl(fd1, PERF_EVENT_IOC_ENABLE, 0);
+        ioctl(fd2, PERF_EVENT_IOC_ENABLE, 0);
+
+        /*
+         * Kick off the test by trigering 'fd1'
+         * breakpoint.
+         */
+        test_function();
+
+        ioctl(fd1, PERF_EVENT_IOC_DISABLE, 0);
+        ioctl(fd2, PERF_EVENT_IOC_DISABLE, 0);
+
+        count1 = bp_count(fd1);
+        count2 = bp_count(fd2);
+
+        close(fd1);
+        close(fd2);
+
+        pr_debug("count1 %lld, count2 %lld, overflow %d\n",
+                 count1, count2, overflows);
+
+        if (count1 != 1) {
+                if (count1 == 11)
+                        pr_debug("failed: RF EFLAG recursion issue detected\n");
+                else
+                        pr_debug("failed: wrong count for bp1%lld\n", count1);
+        }
+
+        if (overflows != 1)
+                pr_debug("failed: wrong overflow hit\n");
+
+        if (count2 != 1)
+                pr_debug("failed: wrong count for bp2\n");
+
+        return count1 == 1 && overflows == 1 && count2 == 1 ?
+                TEST_OK : TEST_FAIL;
+}
diff --git a/tools/perf/tests/bp_signal_overflow.c b/tools/perf/tests/bp_signal_overflow.c
new file mode 100644
index 000000000000..fe7ed28815f8
--- /dev/null
+++ b/tools/perf/tests/bp_signal_overflow.c
@@ -0,0 +1,126 @@
+/*
+ * Originally done by Vince Weaver <vincent.weaver@maine.edu> for
+ * perf_event_tests (git://github.com/deater/perf_event_tests)
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <time.h>
+#include <fcntl.h>
+#include <signal.h>
+#include <sys/mman.h>
+#include <linux/compiler.h>
+#include <linux/hw_breakpoint.h>
+
+#include "tests.h"
+#include "debug.h"
+#include "perf.h"
+
+static int overflows;
+
+__attribute__ ((noinline))
+static int test_function(void)
+{
+        return time(NULL);
+}
+
+static void sig_handler(int signum __maybe_unused,
+                        siginfo_t *oh __maybe_unused,
+                        void *uc __maybe_unused)
+{
+        overflows++;
+}
+
+static long long bp_count(int fd)
+{
+        long long count;
+        int ret;
+
+        ret = read(fd, &count, sizeof(long long));
+        if (ret != sizeof(long long)) {
+                pr_debug("failed to read: %d\n", ret);
+                return TEST_FAIL;
+        }
+
+        return count;
+}
+
+#define EXECUTIONS 10000
+#define THRESHOLD  100
+
+int test__bp_signal_overflow(void)
+{
+        struct perf_event_attr pe;
+        struct sigaction sa;
+        long long count;
+        int fd, i, fails = 0;
+
+        /* setup SIGIO signal handler */
+        memset(&sa, 0, sizeof(struct sigaction));
+        sa.sa_sigaction = (void *) sig_handler;
+        sa.sa_flags = SA_SIGINFO;
+
+        if (sigaction(SIGIO, &sa, NULL) < 0) {
+                pr_debug("failed setting up signal handler\n");
+                return TEST_FAIL;
+        }
+
+        memset(&pe, 0, sizeof(struct perf_event_attr));
+        pe.type = PERF_TYPE_BREAKPOINT;
+        pe.size = sizeof(struct perf_event_attr);
+
+        pe.config = 0;
+        pe.bp_type = HW_BREAKPOINT_X;
+        pe.bp_addr = (unsigned long) test_function;
+        pe.bp_len = sizeof(long);
+
+        pe.sample_period = THRESHOLD;
+        pe.sample_type = PERF_SAMPLE_IP;
+        pe.wakeup_events = 1;
+
+        pe.disabled = 1;
+        pe.exclude_kernel = 1;
+        pe.exclude_hv = 1;
+
+        fd = sys_perf_event_open(&pe, 0, -1, -1, 0);
+        if (fd < 0) {
+                pr_debug("failed opening event %llx\n", pe.config);
+                return TEST_FAIL;
+        }
+
+        fcntl(fd, F_SETFL, O_RDWR|O_NONBLOCK|O_ASYNC);
+        fcntl(fd, F_SETSIG, SIGIO);
+        fcntl(fd, F_SETOWN, getpid());
+
+        ioctl(fd, PERF_EVENT_IOC_RESET, 0);
+        ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);
+
+        for (i = 0; i < EXECUTIONS; i++)
+                test_function();
+
+        ioctl(fd, PERF_EVENT_IOC_DISABLE, 0);
+
+        count = bp_count(fd);
+
+        close(fd);
+
+        pr_debug("count %lld, overflow %d\n",
+                 count, overflows);
+
+        if (count != EXECUTIONS) {
+                pr_debug("\tWrong number of executions %lld != %d\n",
+                count, EXECUTIONS);
+                fails++;
+        }
+
+        if (overflows != EXECUTIONS / THRESHOLD) {
+                pr_debug("\tWrong number of overflows %d != %d\n",
+                overflows, EXECUTIONS / THRESHOLD);
+                fails++;
+        }
+
+        return fails ? TEST_FAIL : TEST_OK;
+}
diff --git a/tools/perf/tests/builtin-test.c b/tools/perf/tests/builtin-test.c
index acb98e0e39f2..0918ada4cc41 100644
--- a/tools/perf/tests/builtin-test.c
+++ b/tools/perf/tests/builtin-test.c
@@ -78,6 +78,22 @@ static struct test {
                 .func = test__python_use,
         },
         {
+                .desc = "Test breakpoint overflow signal handler",
+                .func = test__bp_signal,
+        },
+        {
+                .desc = "Test breakpoint overflow sampling",
+                .func = test__bp_signal_overflow,
+        },
+        {
+                .desc = "Test number of exit event of a simple workload",
+                .func = test__task_exit,
+        },
+        {
+                .desc = "Test software clock events have valid period values",
+                .func = test__sw_clock_freq,
+        },
+        {
                 .func = NULL,
         },
 };
diff --git a/tools/perf/tests/evsel-roundtrip-name.c b/tools/perf/tests/evsel-roundtrip-name.c
index 0fd99a9adb91..0197bda9c461 100644
--- a/tools/perf/tests/evsel-roundtrip-name.c
+++ b/tools/perf/tests/evsel-roundtrip-name.c
@@ -8,7 +8,7 @@ static int perf_evsel__roundtrip_cache_name_test(void)
         char name[128];
         int type, op, err = 0, ret = 0, i, idx;
         struct perf_evsel *evsel;
-        struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+        struct perf_evlist *evlist = perf_evlist__new();
 
         if (evlist == NULL)
                 return -ENOMEM;
@@ -64,7 +64,7 @@ static int __perf_evsel__name_array_test(const char *names[], int nr_names)
 {
         int i, err;
         struct perf_evsel *evsel;
-        struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+        struct perf_evlist *evlist = perf_evlist__new();
 
         if (evlist == NULL)
                 return -ENOMEM;
diff --git a/tools/perf/tests/hists_link.c b/tools/perf/tests/hists_link.c
index 1be64a6c5daf..89085a9615e2 100644
--- a/tools/perf/tests/hists_link.c
+++ b/tools/perf/tests/hists_link.c
@@ -223,7 +223,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine)
                                                           &sample, 0) < 0)
                                 goto out;
 
-                        he = __hists__add_entry(&evsel->hists, &al, NULL, 1);
+                        he = __hists__add_entry(&evsel->hists, &al, NULL, 1, 1);
                         if (he == NULL)
                                 goto out;
 
@@ -247,7 +247,7 @@ static int add_hist_entries(struct perf_evlist *evlist, struct machine *machine)
                                                           &sample, 0) < 0)
                                 goto out;
 
-                        he = __hists__add_entry(&evsel->hists, &al, NULL, 1);
+                        he = __hists__add_entry(&evsel->hists, &al, NULL, 1, 1);
                         if (he == NULL)
                                 goto out;
 
@@ -436,7 +436,7 @@ int test__hists_link(void)
         struct machines machines;
         struct machine *machine = NULL;
         struct perf_evsel *evsel, *first;
-        struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+        struct perf_evlist *evlist = perf_evlist__new();
 
         if (evlist == NULL)
                 return -ENOMEM;
diff --git a/tools/perf/tests/mmap-basic.c b/tools/perf/tests/mmap-basic.c
index cdd50755af51..5b1b5aba722b 100644
--- a/tools/perf/tests/mmap-basic.c
+++ b/tools/perf/tests/mmap-basic.c
@@ -53,12 +53,14 @@ int test__basic_mmap(void)
                 goto out_free_cpus;
         }
 
-        evlist = perf_evlist__new(cpus, threads);
+        evlist = perf_evlist__new();
         if (evlist == NULL) {
                 pr_debug("perf_evlist__new\n");
                 goto out_free_cpus;
         }
 
+        perf_evlist__set_maps(evlist, cpus, threads);
+
         for (i = 0; i < nsyscalls; ++i) {
                 char name[64];
 
diff --git a/tools/perf/tests/open-syscall-tp-fields.c b/tools/perf/tests/open-syscall-tp-fields.c
index 1c52fdc1164e..fc5b9fca8b47 100644
--- a/tools/perf/tests/open-syscall-tp-fields.c
+++ b/tools/perf/tests/open-syscall-tp-fields.c
@@ -18,7 +18,7 @@ int test__syscall_open_tp_fields(void)
         };
         const char *filename = "/etc/passwd";
         int flags = O_RDONLY | O_DIRECTORY;
-        struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+        struct perf_evlist *evlist = perf_evlist__new();
         struct perf_evsel *evsel;
         int err = -1, i, nr_events = 0, nr_polls = 0;
 
@@ -48,13 +48,13 @@ int test__syscall_open_tp_fields(void)
         err = perf_evlist__open(evlist);
         if (err < 0) {
                 pr_debug("perf_evlist__open: %s\n", strerror(errno));
-                goto out_delete_evlist;
+                goto out_delete_maps;
         }
 
         err = perf_evlist__mmap(evlist, UINT_MAX, false);
         if (err < 0) {
                 pr_debug("perf_evlist__mmap: %s\n", strerror(errno));
-                goto out_delete_evlist;
+                goto out_close_evlist;
         }
 
         perf_evlist__enable(evlist);
@@ -110,6 +110,10 @@ out_ok:
         err = 0;
 out_munmap:
         perf_evlist__munmap(evlist);
+out_close_evlist:
+        perf_evlist__close(evlist);
+out_delete_maps:
+        perf_evlist__delete_maps(evlist);
 out_delete_evlist:
         perf_evlist__delete(evlist);
 out:
diff --git a/tools/perf/tests/parse-events.c b/tools/perf/tests/parse-events.c
index c5636f36fe31..88e2f44cb157 100644
--- a/tools/perf/tests/parse-events.c
+++ b/tools/perf/tests/parse-events.c
@@ -3,7 +3,7 @@
 #include "evsel.h"
 #include "evlist.h"
 #include "sysfs.h"
-#include "debugfs.h"
+#include <lk/debugfs.h>
 #include "tests.h"
 #include <linux/hw_breakpoint.h>
 
@@ -1218,7 +1218,7 @@ static int test_event(struct evlist_test *e)
         struct perf_evlist *evlist;
         int ret;
 
-        evlist = perf_evlist__new(NULL, NULL);
+        evlist = perf_evlist__new();
         if (evlist == NULL)
                 return -ENOMEM;
 
diff --git a/tools/perf/tests/perf-record.c b/tools/perf/tests/perf-record.c
index 1e8e5128d0da..72d8881873b0 100644
--- a/tools/perf/tests/perf-record.c
+++ b/tools/perf/tests/perf-record.c
@@ -45,7 +45,7 @@ int test__PERF_RECORD(void)
         };
         cpu_set_t cpu_mask;
         size_t cpu_mask_size = sizeof(cpu_mask);
-        struct perf_evlist *evlist = perf_evlist__new(NULL, NULL);
+        struct perf_evlist *evlist = perf_evlist__new();
         struct perf_evsel *evsel;
         struct perf_sample sample;
         const char *cmd = "sleep";
@@ -93,7 +93,8 @@ int test__PERF_RECORD(void)
          * so that we have time to open the evlist (calling sys_perf_event_open
          * on all the fds) and then mmap them.
          */
-        err = perf_evlist__prepare_workload(evlist, &opts, argv);
+        err = perf_evlist__prepare_workload(evlist, &opts.target, argv,
+                                            false, false);
         if (err < 0) {
                 pr_debug("Couldn't run the workload!\n");
                 goto out_delete_maps;
@@ -142,7 +143,7 @@ int test__PERF_RECORD(void)
         err = perf_evlist__mmap(evlist, opts.mmap_pages, false);
         if (err < 0) {
                 pr_debug("perf_evlist__mmap: %s\n", strerror(errno));
-                goto out_delete_maps;
+                goto out_close_evlist;
         }
 
         /*
@@ -305,6 +306,8 @@ found_exit:
         }
 out_err:
         perf_evlist__munmap(evlist);
+out_close_evlist:
+        perf_evlist__close(evlist);
 out_delete_maps:
         perf_evlist__delete_maps(evlist);
 out_delete_evlist:
diff --git a/tools/perf/tests/sw-clock.c b/tools/perf/tests/sw-clock.c
new file mode 100644
index 000000000000..2e41e2d32ccc
--- /dev/null
+++ b/tools/perf/tests/sw-clock.c
@@ -0,0 +1,119 @@
+#include <unistd.h>
+#include <stdlib.h>
+#include <signal.h>
+#include <sys/mman.h>
+
+#include "tests.h"
+#include "util/evsel.h"
+#include "util/evlist.h"
+#include "util/cpumap.h"
+#include "util/thread_map.h"
+
+#define NR_LOOPS  1000000
+
+/*
+ * This test will open software clock events (cpu-clock, task-clock)
+ * then check their frequency -> period conversion has no artifact of
+ * setting period to 1 forcefully.
+ */
+static int __test__sw_clock_freq(enum perf_sw_ids clock_id)
+{
+        int i, err = -1;
+        volatile int tmp = 0;
+        u64 total_periods = 0;
+        int nr_samples = 0;
+        union perf_event *event;
+        struct perf_evsel *evsel;
+        struct perf_evlist *evlist;
+        struct perf_event_attr attr = {
+                .type = PERF_TYPE_SOFTWARE,
+                .config = clock_id,
+                .sample_type = PERF_SAMPLE_PERIOD,
+                .exclude_kernel = 1,
+                .disabled = 1,
+                .freq = 1,
+        };
+
+        attr.sample_freq = 10000;
+
+        evlist = perf_evlist__new();
+        if (evlist == NULL) {
+                pr_debug("perf_evlist__new\n");
+                return -1;
+        }
+
+        evsel = perf_evsel__new(&attr, 0);
+        if (evsel == NULL) {
+                pr_debug("perf_evsel__new\n");
+                goto out_free_evlist;
+        }
+        perf_evlist__add(evlist, evsel);
+
+        evlist->cpus = cpu_map__dummy_new();
+        evlist->threads = thread_map__new_by_tid(getpid());
+        if (!evlist->cpus || !evlist->threads) {
+                err = -ENOMEM;
+                pr_debug("Not enough memory to create thread/cpu maps\n");
+                goto out_delete_maps;
+        }
+
+        perf_evlist__open(evlist);
+
+        err = perf_evlist__mmap(evlist, 128, true);
+        if (err < 0) {
+                pr_debug("failed to mmap event: %d (%s)\n", errno,
+                         strerror(errno));
+                goto out_close_evlist;
+        }
+
+        perf_evlist__enable(evlist);
+
+        /* collect samples */
+        for (i = 0; i < NR_LOOPS; i++)
+                tmp++;
+
+        perf_evlist__disable(evlist);
+
+        while ((event = perf_evlist__mmap_read(evlist, 0)) != NULL) {
+                struct perf_sample sample;
+
+                if (event->header.type != PERF_RECORD_SAMPLE)
+                        continue;
+
+                err = perf_evlist__parse_sample(evlist, event, &sample);
+                if (err < 0) {
+                        pr_debug("Error during parse sample\n");
+                        goto out_unmap_evlist;
+                }
+
+                total_periods += sample.period;
+                nr_samples++;
+        }
+
+        if ((u64) nr_samples == total_periods) {
+                pr_debug("All (%d) samples have period value of 1!\n",
+                         nr_samples);
+                err = -1;
+        }
+
+out_unmap_evlist:
+        perf_evlist__munmap(evlist);
+out_close_evlist:
+        perf_evlist__close(evlist);
+out_delete_maps:
+        perf_evlist__delete_maps(evlist);
+out_free_evlist:
+        perf_evlist__delete(evlist);
+        return err;
+}
+
+int test__sw_clock_freq(void)
+{
+        int ret;
+
+        ret = __test__sw_clock_freq(PERF_COUNT_SW_CPU_CLOCK);
+        if (!ret)
+                ret = __test__sw_clock_freq(PERF_COUNT_SW_TASK_CLOCK);
+
+        return ret;
+}
diff --git a/tools/perf/tests/task-exit.c b/tools/perf/tests/task-exit.c
new file mode 100644
index 000000000000..28fe5894b061
--- /dev/null
+++ b/tools/perf/tests/task-exit.c
@@ -0,0 +1,123 @@
+#include "evlist.h"
+#include "evsel.h"
+#include "thread_map.h"
+#include "cpumap.h"
+#include "tests.h"
+
+#include <signal.h>
+
+static int exited;
+static int nr_exit;
+
+static void sig_handler(int sig)
+{
+        exited = 1;
+
+        if (sig == SIGUSR1)
+                nr_exit = -1;
+}
+
+/*
+ * This test will start a workload that does nothing then it checks
+ * if the number of exit event reported by the kernel is 1 or not
+ * in order to check the kernel returns correct number of event.
+ */
+int test__task_exit(void)
+{
+        int err = -1;
+        union perf_event *event;
+        struct perf_evsel *evsel;
+        struct perf_evlist *evlist;
+        struct perf_target target = {
+                .uid            = UINT_MAX,
+                .uses_mmap      = true,
+        };
+        const char *argv[] = { "true", NULL };
+
+        signal(SIGCHLD, sig_handler);
+        signal(SIGUSR1, sig_handler);
+
+        evlist = perf_evlist__new();
+        if (evlist == NULL) {
+                pr_debug("perf_evlist__new\n");
+                return -1;
+        }
+        /*
+         * We need at least one evsel in the evlist, use the default
+         * one: "cycles".
+         */
+        err = perf_evlist__add_default(evlist);
+        if (err < 0) {
+                pr_debug("Not enough memory to create evsel\n");
+                goto out_free_evlist;
+        }
+
+        /*
+         * Create maps of threads and cpus to monitor. In this case
+         * we start with all threads and cpus (-1, -1) but then in
+         * perf_evlist__prepare_workload we'll fill in the only thread
+         * we're monitoring, the one forked there.
+         */
+        evlist->cpus = cpu_map__dummy_new();
+        evlist->threads = thread_map__new_by_tid(-1);
+        if (!evlist->cpus || !evlist->threads) {
+                err = -ENOMEM;
+                pr_debug("Not enough memory to create thread/cpu maps\n");
+                goto out_delete_maps;
+        }
+
+        err = perf_evlist__prepare_workload(evlist, &target, argv, false, true);
+        if (err < 0) {
+                pr_debug("Couldn't run the workload!\n");
+                goto out_delete_maps;
+        }
+
+        evsel = perf_evlist__first(evlist);
+        evsel->attr.task = 1;
+        evsel->attr.sample_freq = 0;
+        evsel->attr.inherit = 0;
+        evsel->attr.watermark = 0;
+        evsel->attr.wakeup_events = 1;
+        evsel->attr.exclude_kernel = 1;
+
+        err = perf_evlist__open(evlist);
+        if (err < 0) {
+                pr_debug("Couldn't open the evlist: %s\n", strerror(-err));
+                goto out_delete_maps;
+        }
+
+        if (perf_evlist__mmap(evlist, 128, true) < 0) {
+                pr_debug("failed to mmap events: %d (%s)\n", errno,
+                         strerror(errno));
+                goto out_close_evlist;
+        }
+
+        perf_evlist__start_workload(evlist);
+
+retry:
+        while ((event = perf_evlist__mmap_read(evlist, 0)) != NULL) {
+                if (event->header.type != PERF_RECORD_EXIT)
+                        continue;
+
+                nr_exit++;
+        }
+
+        if (!exited || !nr_exit) {
+                poll(evlist->pollfd, evlist->nr_fds, -1);
+                goto retry;
+        }
+
+        if (nr_exit != 1) {
+                pr_debug("received %d EXIT records\n", nr_exit);
+                err = -1;
+        }
+
+        perf_evlist__munmap(evlist);
+out_close_evlist:
+        perf_evlist__close(evlist);
+out_delete_maps:
+        perf_evlist__delete_maps(evlist);
+out_free_evlist:
+        perf_evlist__delete(evlist);
+        return err;
+}
diff --git a/tools/perf/tests/tests.h b/tools/perf/tests/tests.h
index 5de0be1ff4b6..dd7feae2d37b 100644
--- a/tools/perf/tests/tests.h
+++ b/tools/perf/tests/tests.h
@@ -23,5 +23,9 @@ int test__dso_data(void);
 int test__parse_events(void);
 int test__hists_link(void);
 int test__python_use(void);
+int test__bp_signal(void);
+int test__bp_signal_overflow(void);
+int test__task_exit(void);
+int test__sw_clock_freq(void);
 
 #endif /* TESTS_H */
diff --git a/tools/perf/ui/browser.c b/tools/perf/ui/browser.c
index 809ea4632a34..bbc782e364b0 100644
--- a/tools/perf/ui/browser.c
+++ b/tools/perf/ui/browser.c
@@ -2,7 +2,6 @@
 #include "../cache.h"
 #include "../../perf.h"
 #include "libslang.h"
-#include <newt.h>
 #include "ui.h"
 #include "util.h"
 #include <linux/compiler.h>
@@ -234,7 +233,7 @@ void ui_browser__reset_index(struct ui_browser *browser)
 void __ui_browser__show_title(struct ui_browser *browser, const char *title)
 {
         SLsmg_gotorc(0, 0);
-        ui_browser__set_color(browser, NEWT_COLORSET_ROOT);
+        ui_browser__set_color(browser, HE_COLORSET_ROOT);
         slsmg_write_nstring(title, browser->width + 1);
 }
 
@@ -514,6 +513,12 @@ static struct ui_browser_colorset {
                 .bg       = "default",
         },
         {
+                .colorset = HE_COLORSET_ROOT,
+                .name     = "root",
+                .fg       = "white",
+                .bg       = "blue",
+        },
+        {
                 .name = NULL,
         }
 };
diff --git a/tools/perf/ui/browser.h b/tools/perf/ui/browser.h
index af70314605e5..404ff66a3e36 100644
--- a/tools/perf/ui/browser.h
+++ b/tools/perf/ui/browser.h
@@ -11,6 +11,7 @@
 #define HE_COLORSET_SELECTED    53
 #define HE_COLORSET_CODE        54
 #define HE_COLORSET_ADDR        55
+#define HE_COLORSET_ROOT        56
 
 struct ui_browser {
         u64           index, top_idx;
diff --git a/tools/perf/ui/browsers/annotate.c b/tools/perf/ui/browsers/annotate.c
index 7dca1555c610..cc64d3f7fc36 100644
--- a/tools/perf/ui/browsers/annotate.c
+++ b/tools/perf/ui/browsers/annotate.c
@@ -8,15 +8,19 @@
 #include "../../util/hist.h"
 #include "../../util/sort.h"
 #include "../../util/symbol.h"
+#include "../../util/evsel.h"
 #include <pthread.h>
-#include <newt.h>
 
 struct browser_disasm_line {
         struct rb_node  rb_node;
-        double          percent;
         u32             idx;
         int             idx_asm;
         int             jump_sources;
+        /*
+         * actual length of this array is saved on the nr_events field
+         * of the struct annotate_browser
+         */
+        double          percent[1];
 };
 
 static struct annotate_browser_opt {
@@ -33,8 +37,9 @@ struct annotate_browser {
         struct ui_browser b;
         struct rb_root    entries;
         struct rb_node    *curr_hot;
-        struct disasm_line        *selection;
+        struct disasm_line  *selection;
         struct disasm_line  **offsets;
+        int                 nr_events;
         u64                 start;
         int                 nr_asm_entries;
         int                 nr_entries;
@@ -94,14 +99,24 @@ static void annotate_browser__write(struct ui_browser *browser, void *entry, int
                              (!current_entry || (browser->use_navkeypressed &&
                                                  !browser->navkeypressed)));
         int width = browser->width, printed;
+        int i, pcnt_width = 7 * ab->nr_events;
+        double percent_max = 0.0;
         char bf[256];
 
-        if (dl->offset != -1 && bdl->percent != 0.0) {
-                ui_browser__set_percent_color(browser, bdl->percent, current_entry);
-                slsmg_printf("%6.2f ", bdl->percent);
+        for (i = 0; i < ab->nr_events; i++) {
+                if (bdl->percent[i] > percent_max)
+                        percent_max = bdl->percent[i];
+        }
+
+        if (dl->offset != -1 && percent_max != 0.0) {
+                for (i = 0; i < ab->nr_events; i++) {
+                        ui_browser__set_percent_color(browser, bdl->percent[i],
+                                                      current_entry);
+                        slsmg_printf("%6.2f ", bdl->percent[i]);
+                }
         } else {
                 ui_browser__set_percent_color(browser, 0, current_entry);
-                slsmg_write_nstring(" ", 7);
+                slsmg_write_nstring(" ", pcnt_width);
         }
 
         SLsmg_write_char(' ');
@@ -111,12 +126,12 @@ static void annotate_browser__write(struct ui_browser *browser, void *entry, int
                 width += 1;
 
         if (!*dl->line)
-                slsmg_write_nstring(" ", width - 7);
+                slsmg_write_nstring(" ", width - pcnt_width);
         else if (dl->offset == -1) {
                 printed = scnprintf(bf, sizeof(bf), "%*s  ",
                                     ab->addr_width, " ");
                 slsmg_write_nstring(bf, printed);
-                slsmg_write_nstring(dl->line, width - printed - 6);
+                slsmg_write_nstring(dl->line, width - printed - pcnt_width + 1);
         } else {
                 u64 addr = dl->offset;
                 int color = -1;
@@ -175,7 +190,7 @@ static void annotate_browser__write(struct ui_browser *browser, void *entry, int
                 }
 
                 disasm_line__scnprintf(dl, bf, sizeof(bf), !annotate_browser__opts.use_offset);
-                slsmg_write_nstring(bf, width - 10 - printed);
+                slsmg_write_nstring(bf, width - pcnt_width - 3 - printed);
         }
 
         if (current_entry)
@@ -200,6 +215,7 @@ static void annotate_browser__draw_current_jump(struct ui_browser *browser)
         unsigned int from, to;
         struct map_symbol *ms = ab->b.priv;
         struct symbol *sym = ms->sym;
+        u8 pcnt_width = 7;
 
         /* PLT symbols contain external offsets */
         if (strstr(sym->name, "@plt"))
@@ -223,57 +239,44 @@ static void annotate_browser__draw_current_jump(struct ui_browser *browser)
                 to = (u64)btarget->idx;
         }
 
+        pcnt_width *= ab->nr_events;
+
         ui_browser__set_color(browser, HE_COLORSET_CODE);
-        __ui_browser__line_arrow(browser, 9 + ab->addr_width, from, to);
+        __ui_browser__line_arrow(browser, pcnt_width + 2 + ab->addr_width,
+                                 from, to);
 }
 
 static unsigned int annotate_browser__refresh(struct ui_browser *browser)
 {
+        struct annotate_browser *ab = container_of(browser, struct annotate_browser, b);
         int ret = ui_browser__list_head_refresh(browser);
+        int pcnt_width;
+
+        pcnt_width = 7 * ab->nr_events;
 
         if (annotate_browser__opts.jump_arrows)
                 annotate_browser__draw_current_jump(browser);
 
         ui_browser__set_color(browser, HE_COLORSET_NORMAL);
-        __ui_browser__vline(browser, 7, 0, browser->height - 1);
+        __ui_browser__vline(browser, pcnt_width, 0, browser->height - 1);
         return ret;
 }
 
-static double disasm_line__calc_percent(struct disasm_line *dl, struct symbol *sym, int evidx)
+static int disasm__cmp(struct browser_disasm_line *a,
+                       struct browser_disasm_line *b, int nr_pcnt)
 {
-        double percent = 0.0;
-
-        if (dl->offset != -1) {
-                int len = sym->end - sym->start;
-                unsigned int hits = 0;
-                struct annotation *notes = symbol__annotation(sym);
-                struct source_line *src_line = notes->src->lines;
-                struct sym_hist *h = annotation__histogram(notes, evidx);
-                s64 offset = dl->offset;
-                struct disasm_line *next;
+        int i;
 
-                next = disasm__get_next_ip_line(&notes->src->source, dl);
-                while (offset < (s64)len &&
-                       (next == NULL || offset < next->offset)) {
-                        if (src_line) {
-                                percent += src_line[offset].percent;
-                        } else
-                                hits += h->addr[offset];
-
-                        ++offset;
-                }
-                /*
-                 * If the percentage wasn't already calculated in
-                 * symbol__get_source_line, do it now:
-                 */
-                if (src_line == NULL && h->sum)
-                        percent = 100.0 * hits / h->sum;
+        for (i = 0; i < nr_pcnt; i++) {
+                if (a->percent[i] == b->percent[i])
+                        continue;
+                return a->percent[i] < b->percent[i];
         }
-
-        return percent;
+        return 0;
 }
 
-static void disasm_rb_tree__insert(struct rb_root *root, struct browser_disasm_line *bdl)
+static void disasm_rb_tree__insert(struct rb_root *root, struct browser_disasm_line *bdl,
+                                   int nr_events)
 {
         struct rb_node **p = &root->rb_node;
         struct rb_node *parent = NULL;
@@ -282,7 +285,8 @@ static void disasm_rb_tree__insert(struct rb_root *root, struct browser_disasm_l
         while (*p != NULL) {
                 parent = *p;
                 l = rb_entry(parent, struct browser_disasm_line, rb_node);
-                if (bdl->percent < l->percent)
+
+                if (disasm__cmp(bdl, l, nr_events))
                         p = &(*p)->rb_left;
                 else
                         p = &(*p)->rb_right;
@@ -331,12 +335,13 @@ static void annotate_browser__set_rb_top(struct annotate_browser *browser,
 }
 
 static void annotate_browser__calc_percent(struct annotate_browser *browser,
-                                           int evidx)
+                                           struct perf_evsel *evsel)
 {
         struct map_symbol *ms = browser->b.priv;
         struct symbol *sym = ms->sym;
         struct annotation *notes = symbol__annotation(sym);
-        struct disasm_line *pos;
+        struct disasm_line *pos, *next;
+        s64 len = symbol__size(sym);
 
         browser->entries = RB_ROOT;
 
@@ -344,12 +349,34 @@ static void annotate_browser__calc_percent(struct annotate_browser *browser,
 
         list_for_each_entry(pos, &notes->src->source, node) {
                 struct browser_disasm_line *bpos = disasm_line__browser(pos);
-                bpos->percent = disasm_line__calc_percent(pos, sym, evidx);
-                if (bpos->percent < 0.01) {
+                const char *path = NULL;
+                double max_percent = 0.0;
+                int i;
+
+                if (pos->offset == -1) {
+                        RB_CLEAR_NODE(&bpos->rb_node);
+                        continue;
+                }
+
+                next = disasm__get_next_ip_line(&notes->src->source, pos);
+
+                for (i = 0; i < browser->nr_events; i++) {
+                        bpos->percent[i] = disasm__calc_percent(notes,
+                                                evsel->idx + i,
+                                                pos->offset,
+                                                next ? next->offset : len,
+                                                &path);
+
+                        if (max_percent < bpos->percent[i])
+                                max_percent = bpos->percent[i];
+                }
+
+                if (max_percent < 0.01) {
                         RB_CLEAR_NODE(&bpos->rb_node);
                         continue;
                 }
-                disasm_rb_tree__insert(&browser->entries, bpos);
+                disasm_rb_tree__insert(&browser->entries, bpos,
+                                       browser->nr_events);
         }
         pthread_mutex_unlock(&notes->lock);
 
@@ -401,7 +428,8 @@ static void annotate_browser__init_asm_mode(struct annotate_browser *browser)
         browser->b.nr_entries = browser->nr_asm_entries;
 }
 
-static bool annotate_browser__callq(struct annotate_browser *browser, int evidx,
+static bool annotate_browser__callq(struct annotate_browser *browser,
+                                    struct perf_evsel *evsel,
                                     struct hist_browser_timer *hbt)
 {
         struct map_symbol *ms = browser->b.priv;
@@ -432,7 +460,7 @@ static bool annotate_browser__callq(struct annotate_browser *browser, int evidx,
         }
 
         pthread_mutex_unlock(&notes->lock);
-        symbol__tui_annotate(target, ms->map, evidx, hbt);
+        symbol__tui_annotate(target, ms->map, evsel, hbt);
         ui_browser__show_title(&browser->b, sym->name);
         return true;
 }
@@ -615,7 +643,8 @@ static void annotate_browser__update_addr_width(struct annotate_browser *browser
                 browser->addr_width += browser->jumps_width + 1;
 }
 
-static int annotate_browser__run(struct annotate_browser *browser, int evidx,
+static int annotate_browser__run(struct annotate_browser *browser,
+                                 struct perf_evsel *evsel,
                                  struct hist_browser_timer *hbt)
 {
         struct rb_node *nd = NULL;
@@ -628,7 +657,7 @@ static int annotate_browser__run(struct annotate_browser *browser, int evidx,
         if (ui_browser__show(&browser->b, sym->name, help) < 0)
                 return -1;
 
-        annotate_browser__calc_percent(browser, evidx);
+        annotate_browser__calc_percent(browser, evsel);
 
         if (browser->curr_hot) {
                 annotate_browser__set_rb_top(browser, browser->curr_hot);
@@ -641,7 +670,7 @@ static int annotate_browser__run(struct annotate_browser *browser, int evidx,
                 key = ui_browser__run(&browser->b, delay_secs);
 
                 if (delay_secs != 0) {
-                        annotate_browser__calc_percent(browser, evidx);
+                        annotate_browser__calc_percent(browser, evsel);
                         /*
                          * Current line focus got out of the list of most active
                          * lines, NULL it so that if TAB|UNTAB is pressed, we
@@ -657,7 +686,7 @@ static int annotate_browser__run(struct annotate_browser *browser, int evidx,
                                 hbt->timer(hbt->arg);
 
                         if (delay_secs != 0)
-                                symbol__annotate_decay_histogram(sym, evidx);
+                                symbol__annotate_decay_histogram(sym, evsel->idx);
                         continue;
                 case K_TAB:
                         if (nd != NULL) {
@@ -754,7 +783,7 @@ show_help:
                                         goto show_sup_ins;
                                 goto out;
                         } else if (!(annotate_browser__jump(browser) ||
-                                     annotate_browser__callq(browser, evidx, hbt))) {
+                                     annotate_browser__callq(browser, evsel, hbt))) {
 show_sup_ins:
                                 ui_helpline__puts("Actions are only available for 'callq', 'retq' & jump instructions.");
                         }
@@ -776,10 +805,10 @@ out:
         return key;
 }
 
-int hist_entry__tui_annotate(struct hist_entry *he, int evidx,
+int hist_entry__tui_annotate(struct hist_entry *he, struct perf_evsel *evsel,
                              struct hist_browser_timer *hbt)
 {
-        return symbol__tui_annotate(he->ms.sym, he->ms.map, evidx, hbt);
+        return symbol__tui_annotate(he->ms.sym, he->ms.map, evsel, hbt);
 }
 
 static void annotate_browser__mark_jump_targets(struct annotate_browser *browser,
@@ -826,7 +855,8 @@ static inline int width_jumps(int n)
         return 1;
 }
 
-int symbol__tui_annotate(struct symbol *sym, struct map *map, int evidx,
+int symbol__tui_annotate(struct symbol *sym, struct map *map,
+                         struct perf_evsel *evsel,
                          struct hist_browser_timer *hbt)
 {
         struct disasm_line *pos, *n;
@@ -847,6 +877,8 @@ int symbol__tui_annotate(struct symbol *sym, struct map *map, int evidx,
                 },
         };
         int ret = -1;
+        int nr_pcnt = 1;
+        size_t sizeof_bdl = sizeof(struct browser_disasm_line);
 
         if (sym == NULL)
                 return -1;
@@ -862,7 +894,12 @@ int symbol__tui_annotate(struct symbol *sym, struct map *map, int evidx,
                 return -1;
         }
 
-        if (symbol__annotate(sym, map, sizeof(struct browser_disasm_line)) < 0) {
+        if (perf_evsel__is_group_event(evsel)) {
+                nr_pcnt = evsel->nr_members;
+                sizeof_bdl += sizeof(double) * (nr_pcnt - 1);
+        }
+
+        if (symbol__annotate(sym, map, sizeof_bdl) < 0) {
                 ui__error("%s", ui_helpline__last_msg);
                 goto out_free_offsets;
         }
@@ -900,6 +937,7 @@ int symbol__tui_annotate(struct symbol *sym, struct map *map, int evidx,
         browser.addr_width = browser.target_width = browser.min_addr_width = hex_width(size);
         browser.max_addr_width = hex_width(sym->end);
         browser.jumps_width = width_jumps(browser.max_jump_sources);
+        browser.nr_events = nr_pcnt;
         browser.b.nr_entries = browser.nr_entries;
         browser.b.entries = &notes->src->source,
         browser.b.width += 18; /* Percentage */
@@ -909,7 +947,7 @@ int symbol__tui_annotate(struct symbol *sym, struct map *map, int evidx,
 
         annotate_browser__update_addr_width(&browser);
 
-        ret = annotate_browser__run(&browser, evidx, hbt);
+        ret = annotate_browser__run(&browser, evsel, hbt);
         list_for_each_entry_safe(pos, n, &notes->src->source, node) {
                 list_del(&pos->node);
                 disasm_line__free(pos);
diff --git a/tools/perf/ui/browsers/hists.c b/tools/perf/ui/browsers/hists.c
index aa22704047d6..d88a2d0acb6d 100644
--- a/tools/perf/ui/browsers/hists.c
+++ b/tools/perf/ui/browsers/hists.c
@@ -2,7 +2,6 @@
 #include "../libslang.h"
 #include <stdlib.h>
 #include <string.h>
-#include <newt.h>
 #include <linux/rbtree.h>
 
 #include "../../util/evsel.h"
@@ -1193,7 +1192,7 @@ static int hists__browser_title(struct hists *hists, char *bf, size_t size,
         char buf[512];
         size_t buflen = sizeof(buf);
 
-        if (symbol_conf.event_group && evsel->nr_members > 1) {
+        if (perf_evsel__is_group_event(evsel)) {
                 struct perf_evsel *pos;
 
                 perf_evsel__group_desc(evsel, buf, buflen);
@@ -1599,7 +1598,7 @@ do_annotate:
                          * Don't let this be freed, say, by hists__decay_entry.
                          */
                         he->used = true;
-                        err = hist_entry__tui_annotate(he, evsel->idx, hbt);
+                        err = hist_entry__tui_annotate(he, evsel, hbt);
                         he->used = false;
                         /*
                          * offer option to annotate the other branch source or target
@@ -1709,7 +1708,7 @@ static void perf_evsel_menu__write(struct ui_browser *browser,
         ui_browser__set_color(browser, current_entry ? HE_COLORSET_SELECTED :
                                                        HE_COLORSET_NORMAL);
 
-        if (symbol_conf.event_group && evsel->nr_members > 1) {
+        if (perf_evsel__is_group_event(evsel)) {
                 struct perf_evsel *pos;
 
                 ev_name = perf_evsel__group_name(evsel);
diff --git a/tools/perf/ui/browsers/map.c b/tools/perf/ui/browsers/map.c
index 98851d55a53e..95c7cfb8f2c6 100644
--- a/tools/perf/ui/browsers/map.c
+++ b/tools/perf/ui/browsers/map.c
@@ -1,6 +1,5 @@
 #include "../libslang.h"
 #include <elf.h>
-#include <newt.h>
 #include <inttypes.h>
 #include <sys/ttydefaults.h>
 #include <string.h>
@@ -10,41 +9,9 @@
 #include "../../util/symbol.h"
 #include "../browser.h"
 #include "../helpline.h"
+#include "../keysyms.h"
 #include "map.h"
 
-static int ui_entry__read(const char *title, char *bf, size_t size, int width)
-{
-        struct newtExitStruct es;
-        newtComponent form, entry;
-        const char *result;
-        int err = -1;
-
-        newtCenteredWindow(width, 1, title);
-        form = newtForm(NULL, NULL, 0);
-        if (form == NULL)
-                return -1;
-
-        entry = newtEntry(0, 0, "0x", width, &result, NEWT_FLAG_SCROLL);
-        if (entry == NULL)
-                goto out_free_form;
-
-        newtFormAddComponent(form, entry);
-        newtFormAddHotKey(form, NEWT_KEY_ENTER);
-        newtFormAddHotKey(form, NEWT_KEY_ESCAPE);
-        newtFormAddHotKey(form, NEWT_KEY_LEFT);
-        newtFormAddHotKey(form, CTRL('c'));
-        newtFormRun(form, &es);
-
-        if (result != NULL) {
-                strncpy(bf, result, size);
-                err = 0;
-        }
-out_free_form:
-        newtPopWindow();
-        newtFormDestroy(form);
-        return err;
-}
-
 struct map_browser {
         struct ui_browser b;
         struct map        *map;
@@ -78,10 +45,11 @@ static int map_browser__search(struct map_browser *self)
 {
         char target[512];
         struct symbol *sym;
-        int err = ui_entry__read("Search by name/addr", target, sizeof(target), 40);
-
-        if (err)
-                return err;
+        int err = ui_browser__input_window("Search by name/addr",
+                                           "Prefix with 0x to search by address",
+                                           target, "ENTER: OK, ESC: Cancel", 0);
+        if (err != K_ENTER)
+                return -1;
 
         if (target[0] == '0' && tolower(target[1]) == 'x') {
                 u64 addr = strtoull(target, NULL, 16);
@@ -112,12 +80,20 @@ static int map_browser__run(struct map_browser *self)
         while (1) {
                 key = ui_browser__run(&self->b, 0);
 
-                if (verbose && key == '/')
-                        map_browser__search(self);
-                else
+                switch (key) {
+                case '/':
+                        if (verbose)
+                                map_browser__search(self);
+                default:
                         break;
+                case K_LEFT:
+                case K_ESC:
+                case 'q':
+                case CTRL('c'):
+                        goto out;
+                }
         }
-
+out:
         ui_browser__hide(&self->b);
         return key;
 }
diff --git a/tools/perf/ui/browsers/scripts.c b/tools/perf/ui/browsers/scripts.c
index cbbd44b0d93e..12f009e61e94 100644
--- a/tools/perf/ui/browsers/scripts.c
+++ b/tools/perf/ui/browsers/scripts.c
@@ -1,5 +1,4 @@
 #include <elf.h>
-#include <newt.h>
 #include <inttypes.h>
 #include <sys/ttydefaults.h>
 #include <string.h>
diff --git a/tools/perf/ui/gtk/annotate.c b/tools/perf/ui/gtk/annotate.c
index 7d8dc581a545..f538794615db 100644
--- a/tools/perf/ui/gtk/annotate.c
+++ b/tools/perf/ui/gtk/annotate.c
@@ -1,6 +1,7 @@
 #include "gtk.h"
 #include "util/debug.h"
 #include "util/annotate.h"
+#include "util/evsel.h"
 #include "ui/helpline.h"
 
 
@@ -32,7 +33,7 @@ static int perf_gtk__get_percent(char *buf, size_t size, struct symbol *sym,
                 return 0;
 
         symhist = annotation__histogram(symbol__annotation(sym), evidx);
-        if (!symhist->addr[dl->offset])
+        if (!symbol_conf.event_group && !symhist->addr[dl->offset])
                 return 0;
 
         percent = 100.0 * symhist->addr[dl->offset] / symhist->sum;
@@ -85,7 +86,7 @@ static int perf_gtk__get_line(char *buf, size_t size, struct disasm_line *dl)
 }
 
 static int perf_gtk__annotate_symbol(GtkWidget *window, struct symbol *sym,
-                                struct map *map, int evidx,
+                                struct map *map, struct perf_evsel *evsel,
                                 struct hist_browser_timer *hbt __maybe_unused)
 {
         struct disasm_line *pos, *n;
@@ -118,10 +119,24 @@ static int perf_gtk__annotate_symbol(GtkWidget *window, struct symbol *sym,
 
         list_for_each_entry(pos, &notes->src->source, node) {
                 GtkTreeIter iter;
+                int ret = 0;
 
                 gtk_list_store_append(store, &iter);
 
-                if (perf_gtk__get_percent(s, sizeof(s), sym, pos, evidx))
+                if (perf_evsel__is_group_event(evsel)) {
+                        for (i = 0; i < evsel->nr_members; i++) {
+                                ret += perf_gtk__get_percent(s + ret,
+                                                             sizeof(s) - ret,
+                                                             sym, pos,
+                                                             evsel->idx + i);
+                                ret += scnprintf(s + ret, sizeof(s) - ret, " ");
+                        }
+                } else {
+                        ret = perf_gtk__get_percent(s, sizeof(s), sym, pos,
+                                                    evsel->idx);
+                }
+
+                if (ret)
                         gtk_list_store_set(store, &iter, ANN_COL__PERCENT, s, -1);
                 if (perf_gtk__get_offset(s, sizeof(s), sym, map, pos))
                         gtk_list_store_set(store, &iter, ANN_COL__OFFSET, s, -1);
@@ -139,7 +154,8 @@ static int perf_gtk__annotate_symbol(GtkWidget *window, struct symbol *sym,
         return 0;
 }
 
-int symbol__gtk_annotate(struct symbol *sym, struct map *map, int evidx,
+int symbol__gtk_annotate(struct symbol *sym, struct map *map,
+                         struct perf_evsel *evsel,
                          struct hist_browser_timer *hbt)
 {
         GtkWidget *window;
@@ -206,7 +222,7 @@ int symbol__gtk_annotate(struct symbol *sym, struct map *map, int evidx,
         gtk_notebook_append_page(GTK_NOTEBOOK(notebook), scrolled_window,
                                  tab_label);
 
-        perf_gtk__annotate_symbol(scrolled_window, sym, map, evidx, hbt);
+        perf_gtk__annotate_symbol(scrolled_window, sym, map, evsel, hbt);
         return 0;
 }
 
diff --git a/tools/perf/ui/gtk/hists.c b/tools/perf/ui/gtk/hists.c
index 1e764a8ad259..6f259b3d14e2 100644
--- a/tools/perf/ui/gtk/hists.c
+++ b/tools/perf/ui/gtk/hists.c
@@ -32,21 +32,18 @@ static int __hpp__color_fmt(struct perf_hpp *hpp, struct hist_entry *he,
         int ret;
         double percent = 0.0;
         struct hists *hists = he->hists;
+        struct perf_evsel *evsel = hists_to_evsel(hists);
 
         if (hists->stats.total_period)
                 percent = 100.0 * get_field(he) / hists->stats.total_period;
 
         ret = __percent_color_snprintf(hpp->buf, hpp->size, percent);
 
-        if (symbol_conf.event_group) {
+        if (perf_evsel__is_group_event(evsel)) {
                 int prev_idx, idx_delta;
-                struct perf_evsel *evsel = hists_to_evsel(hists);
                 struct hist_entry *pair;
                 int nr_members = evsel->nr_members;
 
-                if (nr_members <= 1)
-                        return ret;
-
                 prev_idx = perf_evsel__group_idx(evsel);
 
                 list_for_each_entry(pair, &he->pairs.head, pairs.node) {
diff --git a/tools/perf/ui/hist.c b/tools/perf/ui/hist.c
index d671e63aa351..4bf91b09d62d 100644
--- a/tools/perf/ui/hist.c
+++ b/tools/perf/ui/hist.c
@@ -16,6 +16,7 @@ static int __hpp__fmt(struct perf_hpp *hpp, struct hist_entry *he,
 {
         int ret;
         struct hists *hists = he->hists;
+        struct perf_evsel *evsel = hists_to_evsel(hists);
 
         if (fmt_percent) {
                 double percent = 0.0;
@@ -28,15 +29,11 @@ static int __hpp__fmt(struct perf_hpp *hpp, struct hist_entry *he,
         } else
                 ret = print_fn(hpp->buf, hpp->size, fmt, get_field(he));
 
-        if (symbol_conf.event_group) {
+        if (perf_evsel__is_group_event(evsel)) {
                 int prev_idx, idx_delta;
-                struct perf_evsel *evsel = hists_to_evsel(hists);
                 struct hist_entry *pair;
                 int nr_members = evsel->nr_members;
 
-                if (nr_members <= 1)
-                        return ret;
-
                 prev_idx = perf_evsel__group_idx(evsel);
 
                 list_for_each_entry(pair, &he->pairs.head, pairs.node) {
diff --git a/tools/perf/ui/tui/setup.c b/tools/perf/ui/tui/setup.c
index 81efa192e86c..b9401482d110 100644
--- a/tools/perf/ui/tui/setup.c
+++ b/tools/perf/ui/tui/setup.c
@@ -1,4 +1,3 @@
-#include <newt.h>
 #include <signal.h>
 #include <stdbool.h>
 
@@ -88,13 +87,6 @@ int ui__getch(int delay_secs)
         return SLkp_getkey();
 }
 
-static void newt_suspend(void *d __maybe_unused)
-{
-        newtSuspend();
-        raise(SIGTSTP);
-        newtResume();
-}
-
 static void ui__signal(int sig)
 {
         ui__exit(false);
@@ -106,7 +98,17 @@ int ui__init(void)
 {
         int err;
 
-        newtInit();
+        SLutf8_enable(-1);
+        SLtt_get_terminfo();
+        SLtt_get_screen_size();
+
+        err = SLsmg_init_smg();
+        if (err < 0)
+                goto out;
+        err = SLang_init_tty(0, 0, 0);
+        if (err < 0)
+                goto out;
+
         err = SLkp_init();
         if (err < 0) {
                 pr_err("TUI initialization failed.\n");
@@ -115,7 +117,6 @@ int ui__init(void)
 
         SLkp_define_keysym((char *)"^(kB)", SL_KEY_UNTAB);
 
-        newtSetSuspendCallback(newt_suspend, NULL);
         ui_helpline__init();
         ui_browser__init();
         ui_progress__init();
diff --git a/tools/perf/ui/ui.h b/tools/perf/ui/ui.h
index d86359c99907..70cb0d4eb8aa 100644
--- a/tools/perf/ui/ui.h
+++ b/tools/perf/ui/ui.h
@@ -12,7 +12,7 @@ extern int use_browser;
 void setup_browser(bool fallback_to_pager);
 void exit_browser(bool wait_for_ok);
 
-#ifdef NEWT_SUPPORT
+#ifdef SLANG_SUPPORT
 int ui__init(void);
 void ui__exit(bool wait_for_ok);
 #else
diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
index d33fe937e6f1..d102716c43a1 100644
--- a/tools/perf/util/annotate.c
+++ b/tools/perf/util/annotate.c
@@ -14,6 +14,7 @@
 #include "symbol.h"
 #include "debug.h"
 #include "annotate.h"
+#include "evsel.h"
 #include <pthread.h>
 #include <linux/bitops.h>
 
@@ -602,8 +603,42 @@ struct disasm_line *disasm__get_next_ip_line(struct list_head *head, struct disa
         return NULL;
 }
 
+double disasm__calc_percent(struct annotation *notes, int evidx, s64 offset,
+                            s64 end, const char **path)
+{
+        struct source_line *src_line = notes->src->lines;
+        double percent = 0.0;
+
+        if (src_line) {
+                size_t sizeof_src_line = sizeof(*src_line) +
+                                sizeof(src_line->p) * (src_line->nr_pcnt - 1);
+
+                while (offset < end) {
+                        src_line = (void *)notes->src->lines +
+                                        (sizeof_src_line * offset);
+
+                        if (*path == NULL)
+                                *path = src_line->path;
+
+                        percent += src_line->p[evidx].percent;
+                        offset++;
+                }
+        } else {
+                struct sym_hist *h = annotation__histogram(notes, evidx);
+                unsigned int hits = 0;
+
+                while (offset < end)
+                        hits += h->addr[offset++];
+
+                if (h->sum)
+                        percent = 100.0 * hits / h->sum;
+        }
+
+        return percent;
+}
+
 static int disasm_line__print(struct disasm_line *dl, struct symbol *sym, u64 start,
-                      int evidx, u64 len, int min_pcnt, int printed,
+                      struct perf_evsel *evsel, u64 len, int min_pcnt, int printed,
                       int max_lines, struct disasm_line *queue)
 {
         static const char *prev_line;
@@ -611,34 +646,37 @@ static int disasm_line__print(struct disasm_line *dl, struct symbol *sym, u64 st
 
         if (dl->offset != -1) {
                 const char *path = NULL;
-                unsigned int hits = 0;
-                double percent = 0.0;
+                double percent, max_percent = 0.0;
+                double *ppercents = &percent;
+                int i, nr_percent = 1;
                 const char *color;
                 struct annotation *notes = symbol__annotation(sym);
-                struct source_line *src_line = notes->src->lines;
-                struct sym_hist *h = annotation__histogram(notes, evidx);
                 s64 offset = dl->offset;
                 const u64 addr = start + offset;
                 struct disasm_line *next;
 
                 next = disasm__get_next_ip_line(&notes->src->source, dl);
 
-                while (offset < (s64)len &&
-                       (next == NULL || offset < next->offset)) {
-                        if (src_line) {
-                                if (path == NULL)
-                                        path = src_line[offset].path;
-                                percent += src_line[offset].percent;
-                        } else
-                                hits += h->addr[offset];
-
-                        ++offset;
+                if (perf_evsel__is_group_event(evsel)) {
+                        nr_percent = evsel->nr_members;
+                        ppercents = calloc(nr_percent, sizeof(double));
+                        if (ppercents == NULL)
+                                return -1;
                 }
 
-                if (src_line == NULL && h->sum)
-                        percent = 100.0 * hits / h->sum;
+                for (i = 0; i < nr_percent; i++) {
+                        percent = disasm__calc_percent(notes,
+                                        notes->src->lines ? i : evsel->idx + i,
+                                        offset,
+                                        next ? next->offset : (s64) len,
+                                        &path);
+
+                        ppercents[i] = percent;
+                        if (percent > max_percent)
+                                max_percent = percent;
+                }
 
-                if (percent < min_pcnt)
+                if (max_percent < min_pcnt)
                         return -1;
 
                 if (max_lines && printed >= max_lines)
@@ -648,12 +686,12 @@ static int disasm_line__print(struct disasm_line *dl, struct symbol *sym, u64 st
                         list_for_each_entry_from(queue, &notes->src->source, node) {
                                 if (queue == dl)
                                         break;
-                                disasm_line__print(queue, sym, start, evidx, len,
+                                disasm_line__print(queue, sym, start, evsel, len,
                                                     0, 0, 1, NULL);
                         }
                 }
 
-                color = get_percent_color(percent);
+                color = get_percent_color(max_percent);
 
                 /*
                  * Also color the filename and line if needed, with
@@ -669,25 +707,59 @@ static int disasm_line__print(struct disasm_line *dl, struct symbol *sym, u64 st
                         }
                 }
 
-                color_fprintf(stdout, color, " %7.2f", percent);
+                for (i = 0; i < nr_percent; i++) {
+                        percent = ppercents[i];
+                        color = get_percent_color(percent);
+                        color_fprintf(stdout, color, " %7.2f", percent);
+                }
+
                 printf(" :      ");
                 color_fprintf(stdout, PERF_COLOR_MAGENTA, "  %" PRIx64 ":", addr);
                 color_fprintf(stdout, PERF_COLOR_BLUE, "%s\n", dl->line);
+
+                if (ppercents != &percent)
+                        free(ppercents);
+
         } else if (max_lines && printed >= max_lines)
                 return 1;
         else {
+                int width = 8;
+
                 if (queue)
                         return -1;
 
+                if (perf_evsel__is_group_event(evsel))
+                        width *= evsel->nr_members;
+
                 if (!*dl->line)
-                        printf("         :\n");
+                        printf(" %*s:\n", width, " ");
                 else
-                        printf("         :      %s\n", dl->line);
+                        printf(" %*s:   %s\n", width, " ", dl->line);
         }
 
         return 0;
 }
 
+/*
+ * symbol__parse_objdump_line() parses objdump output (with -d --no-show-raw)
+ * which looks like following
+ *
+ *  0000000000415500 <_init>:
+ *    415500:       sub    $0x8,%rsp
+ *    415504:       mov    0x2f5ad5(%rip),%rax        # 70afe0 <_DYNAMIC+0x2f8>
+ *    41550b:       test   %rax,%rax
+ *    41550e:       je     415515 <_init+0x15>
+ *    415510:       callq  416e70 <__gmon_start__@plt>
+ *    415515:       add    $0x8,%rsp
+ *    415519:       retq
+ *
+ * it will be parsed and saved into struct disasm_line as
+ *  <offset>       <name>  <ops.raw>
+ *
+ * The offset will be a relative offset from the start of the symbol and -1
+ * means that it's not a disassembly line so should be treated differently.
+ * The ops.raw part will be parsed further according to type of the instruction.
+ */
 static int symbol__parse_objdump_line(struct symbol *sym, struct map *map,
                                       FILE *file, size_t privsize)
 {
@@ -858,7 +930,7 @@ static void insert_source_line(struct rb_root *root, struct source_line *src_lin
         struct source_line *iter;
         struct rb_node **p = &root->rb_node;
         struct rb_node *parent = NULL;
-        int ret;
+        int i, ret;
 
         while (*p != NULL) {
                 parent = *p;
@@ -866,7 +938,8 @@ static void insert_source_line(struct rb_root *root, struct source_line *src_lin
 
                 ret = strcmp(iter->path, src_line->path);
                 if (ret == 0) {
-                        iter->percent_sum += src_line->percent;
+                        for (i = 0; i < src_line->nr_pcnt; i++)
+                                iter->p[i].percent_sum += src_line->p[i].percent;
                         return;
                 }
 
@@ -876,12 +949,26 @@ static void insert_source_line(struct rb_root *root, struct source_line *src_lin
                         p = &(*p)->rb_right;
         }
 
-        src_line->percent_sum = src_line->percent;
+        for (i = 0; i < src_line->nr_pcnt; i++)
+                src_line->p[i].percent_sum = src_line->p[i].percent;
 
         rb_link_node(&src_line->node, parent, p);
         rb_insert_color(&src_line->node, root);
 }
 
+static int cmp_source_line(struct source_line *a, struct source_line *b)
+{
+        int i;
+
+        for (i = 0; i < a->nr_pcnt; i++) {
+                if (a->p[i].percent_sum == b->p[i].percent_sum)
+                        continue;
+                return a->p[i].percent_sum > b->p[i].percent_sum;
+        }
+
+        return 0;
+}
+
 static void __resort_source_line(struct rb_root *root, struct source_line *src_line)
 {
         struct source_line *iter;
@@ -892,7 +979,7 @@ static void __resort_source_line(struct rb_root *root, struct source_line *src_l
                 parent = *p;
                 iter = rb_entry(parent, struct source_line, node);
 
-                if (src_line->percent_sum > iter->percent_sum)
+                if (cmp_source_line(src_line, iter))
                         p = &(*p)->rb_left;
                 else
                         p = &(*p)->rb_right;
@@ -924,32 +1011,52 @@ static void symbol__free_source_line(struct symbol *sym, int len)
 {
         struct annotation *notes = symbol__annotation(sym);
         struct source_line *src_line = notes->src->lines;
+        size_t sizeof_src_line;
         int i;
 
-        for (i = 0; i < len; i++)
-                free(src_line[i].path);
+        sizeof_src_line = sizeof(*src_line) +
+                          (sizeof(src_line->p) * (src_line->nr_pcnt - 1));
+
+        for (i = 0; i < len; i++) {
+                free(src_line->path);
+                src_line = (void *)src_line + sizeof_src_line;
+        }
 
-        free(src_line);
+        free(notes->src->lines);
         notes->src->lines = NULL;
 }
 
 /* Get the filename:line for the colored entries */
 static int symbol__get_source_line(struct symbol *sym, struct map *map,
-                                   int evidx, struct rb_root *root, int len,
+                                   struct perf_evsel *evsel,
+                                   struct rb_root *root, int len,
                                    const char *filename)
 {
         u64 start;
-        int i;
+        int i, k;
+        int evidx = evsel->idx;
         char cmd[PATH_MAX * 2];
         struct source_line *src_line;
         struct annotation *notes = symbol__annotation(sym);
         struct sym_hist *h = annotation__histogram(notes, evidx);
         struct rb_root tmp_root = RB_ROOT;
+        int nr_pcnt = 1;
+        u64 h_sum = h->sum;
+        size_t sizeof_src_line = sizeof(struct source_line);
+
+        if (perf_evsel__is_group_event(evsel)) {
+                for (i = 1; i < evsel->nr_members; i++) {
+                        h = annotation__histogram(notes, evidx + i);
+                        h_sum += h->sum;
+                }
+                nr_pcnt = evsel->nr_members;
+                sizeof_src_line += (nr_pcnt - 1) * sizeof(src_line->p);
+        }
 
-        if (!h->sum)
+        if (!h_sum)
                 return 0;
 
-        src_line = notes->src->lines = calloc(len, sizeof(struct source_line));
+        src_line = notes->src->lines = calloc(len, sizeof_src_line);
         if (!notes->src->lines)
                 return -1;
 
@@ -960,29 +1067,41 @@ static int symbol__get_source_line(struct symbol *sym, struct map *map,
                 size_t line_len;
                 u64 offset;
                 FILE *fp;
+                double percent_max = 0.0;
 
-                src_line[i].percent = 100.0 * h->addr[i] / h->sum;
-                if (src_line[i].percent <= 0.5)
-                        continue;
+                src_line->nr_pcnt = nr_pcnt;
+
+                for (k = 0; k < nr_pcnt; k++) {
+                        h = annotation__histogram(notes, evidx + k);
+                        src_line->p[k].percent = 100.0 * h->addr[i] / h->sum;
+
+                        if (src_line->p[k].percent > percent_max)
+                                percent_max = src_line->p[k].percent;
+                }
+
+                if (percent_max <= 0.5)
+                        goto next;
 
                 offset = start + i;
                 sprintf(cmd, "addr2line -e %s %016" PRIx64, filename, offset);
                 fp = popen(cmd, "r");
                 if (!fp)
-                        continue;
+                        goto next;
 
                 if (getline(&path, &line_len, fp) < 0 || !line_len)
-                        goto next;
+                        goto next_close;
 
-                src_line[i].path = malloc(sizeof(char) * line_len + 1);
-                if (!src_line[i].path)
-                        goto next;
+                src_line->path = malloc(sizeof(char) * line_len + 1);
+                if (!src_line->path)
+                        goto next_close;
 
-                strcpy(src_line[i].path, path);
-                insert_source_line(&tmp_root, &src_line[i]);
+                strcpy(src_line->path, path);
+                insert_source_line(&tmp_root, src_line);
 
-        next:
+        next_close:
                 pclose(fp);
+        next:
+                src_line = (void *)src_line + sizeof_src_line;
         }
 
         resort_source_line(root, &tmp_root);
@@ -1004,24 +1123,33 @@ static void print_summary(struct rb_root *root, const char *filename)
 
         node = rb_first(root);
         while (node) {
-                double percent;
+                double percent, percent_max = 0.0;
                 const char *color;
                 char *path;
+                int i;
 
                 src_line = rb_entry(node, struct source_line, node);
-                percent = src_line->percent_sum;
-                color = get_percent_color(percent);
+                for (i = 0; i < src_line->nr_pcnt; i++) {
+                        percent = src_line->p[i].percent_sum;
+                        color = get_percent_color(percent);
+                        color_fprintf(stdout, color, " %7.2f", percent);
+
+                        if (percent > percent_max)
+                                percent_max = percent;
+                }
+
                 path = src_line->path;
+                color = get_percent_color(percent_max);
+                color_fprintf(stdout, color, " %s", path);
 
-                color_fprintf(stdout, color, " %7.2f %s", percent, path);
                 node = rb_next(node);
         }
 }
 
-static void symbol__annotate_hits(struct symbol *sym, int evidx)
+static void symbol__annotate_hits(struct symbol *sym, struct perf_evsel *evsel)
 {
         struct annotation *notes = symbol__annotation(sym);
-        struct sym_hist *h = annotation__histogram(notes, evidx);
+        struct sym_hist *h = annotation__histogram(notes, evsel->idx);
         u64 len = symbol__size(sym), offset;
 
         for (offset = 0; offset < len; ++offset)
@@ -1031,9 +1159,9 @@ static void symbol__annotate_hits(struct symbol *sym, int evidx)
         printf("%*s: %" PRIu64 "\n", BITS_PER_LONG / 2, "h->sum", h->sum);
 }
 
-int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx,
-                            bool full_paths, int min_pcnt, int max_lines,
-                            int context)
+int symbol__annotate_printf(struct symbol *sym, struct map *map,
+                            struct perf_evsel *evsel, bool full_paths,
+                            int min_pcnt, int max_lines, int context)
 {
         struct dso *dso = map->dso;
         char *filename;
@@ -1044,6 +1172,8 @@ int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx,
         int printed = 2, queue_len = 0;
         int more = 0;
         u64 len;
+        int width = 8;
+        int namelen;
 
         filename = strdup(dso->long_name);
         if (!filename)
@@ -1055,12 +1185,18 @@ int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx,
                 d_filename = basename(filename);
 
         len = symbol__size(sym);
+        namelen = strlen(d_filename);
+
+        if (perf_evsel__is_group_event(evsel))
+                width *= evsel->nr_members;
 
-        printf(" Percent |      Source code & Disassembly of %s\n", d_filename);
-        printf("------------------------------------------------\n");
+        printf(" %-*.*s|        Source code & Disassembly of %s\n",
+               width, width, "Percent", d_filename);
+        printf("-%-*.*s-------------------------------------\n",
+               width+namelen, width+namelen, graph_dotted_line);
 
         if (verbose)
-                symbol__annotate_hits(sym, evidx);
+                symbol__annotate_hits(sym, evsel);
 
         list_for_each_entry(pos, &notes->src->source, node) {
                 if (context && queue == NULL) {
@@ -1068,7 +1204,7 @@ int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx,
                         queue_len = 0;
                 }
 
-                switch (disasm_line__print(pos, sym, start, evidx, len,
+                switch (disasm_line__print(pos, sym, start, evsel, len,
                                             min_pcnt, printed, max_lines,
                                             queue)) {
                 case 0:
@@ -1163,9 +1299,9 @@ size_t disasm__fprintf(struct list_head *head, FILE *fp)
         return printed;
 }
 
-int symbol__tty_annotate(struct symbol *sym, struct map *map, int evidx,
-                         bool print_lines, bool full_paths, int min_pcnt,
-                         int max_lines)
+int symbol__tty_annotate(struct symbol *sym, struct map *map,
+                         struct perf_evsel *evsel, bool print_lines,
+                         bool full_paths, int min_pcnt, int max_lines)
 {
         struct dso *dso = map->dso;
         const char *filename = dso->long_name;
@@ -1178,12 +1314,12 @@ int symbol__tty_annotate(struct symbol *sym, struct map *map, int evidx,
         len = symbol__size(sym);
 
         if (print_lines) {
-                symbol__get_source_line(sym, map, evidx, &source_line,
+                symbol__get_source_line(sym, map, evsel, &source_line,
                                         len, filename);
                 print_summary(&source_line, filename);
         }
 
-        symbol__annotate_printf(sym, map, evidx, full_paths,
+        symbol__annotate_printf(sym, map, evsel, full_paths,
                                 min_pcnt, max_lines, 0);
         if (print_lines)
                 symbol__free_source_line(sym, len);
diff --git a/tools/perf/util/annotate.h b/tools/perf/util/annotate.h
index c422440fe611..af755156d278 100644
--- a/tools/perf/util/annotate.h
+++ b/tools/perf/util/annotate.h
@@ -50,6 +50,8 @@ bool ins__is_jump(const struct ins *ins);
 bool ins__is_call(const struct ins *ins);
 int ins__scnprintf(struct ins *ins, char *bf, size_t size, struct ins_operands *ops);
 
+struct annotation;
+
 struct disasm_line {
         struct list_head    node;
         s64                 offset;
@@ -68,17 +70,24 @@ void disasm_line__free(struct disasm_line *dl);
 struct disasm_line *disasm__get_next_ip_line(struct list_head *head, struct disasm_line *pos);
 int disasm_line__scnprintf(struct disasm_line *dl, char *bf, size_t size, bool raw);
 size_t disasm__fprintf(struct list_head *head, FILE *fp);
+double disasm__calc_percent(struct annotation *notes, int evidx, s64 offset,
+                            s64 end, const char **path);
 
 struct sym_hist {
         u64             sum;
         u64             addr[0];
 };
 
-struct source_line {
-        struct rb_node  node;
+struct source_line_percent {
         double          percent;
         double          percent_sum;
+};
+
+struct source_line {
+        struct rb_node  node;
         char            *path;
+        int             nr_pcnt;
+        struct source_line_percent p[1];
 };
 
 /** struct annotated_source - symbols with hits have this attached as in sannotation
@@ -130,47 +139,49 @@ void symbol__annotate_zero_histograms(struct symbol *sym);
 
 int symbol__annotate(struct symbol *sym, struct map *map, size_t privsize);
 int symbol__annotate_init(struct map *map __maybe_unused, struct symbol *sym);
-int symbol__annotate_printf(struct symbol *sym, struct map *map, int evidx,
-                            bool full_paths, int min_pcnt, int max_lines,
-                            int context);
+int symbol__annotate_printf(struct symbol *sym, struct map *map,
+                            struct perf_evsel *evsel, bool full_paths,
+                            int min_pcnt, int max_lines, int context);
 void symbol__annotate_zero_histogram(struct symbol *sym, int evidx);
 void symbol__annotate_decay_histogram(struct symbol *sym, int evidx);
 void disasm__purge(struct list_head *head);
 
-int symbol__tty_annotate(struct symbol *sym, struct map *map, int evidx,
-                         bool print_lines, bool full_paths, int min_pcnt,
-                         int max_lines);
+int symbol__tty_annotate(struct symbol *sym, struct map *map,
+                         struct perf_evsel *evsel, bool print_lines,
+                         bool full_paths, int min_pcnt, int max_lines);
 
-#ifdef NEWT_SUPPORT
-int symbol__tui_annotate(struct symbol *sym, struct map *map, int evidx,
+#ifdef SLANG_SUPPORT
+int symbol__tui_annotate(struct symbol *sym, struct map *map,
+                         struct perf_evsel *evsel,
                          struct hist_browser_timer *hbt);
 #else
 static inline int symbol__tui_annotate(struct symbol *sym __maybe_unused,
-                                       struct map *map __maybe_unused,
-                                       int evidx __maybe_unused,
-                                       struct hist_browser_timer *hbt
-                                       __maybe_unused)
+                                struct map *map __maybe_unused,
+                                struct perf_evsel *evsel  __maybe_unused,
+                                struct hist_browser_timer *hbt
+                                __maybe_unused)
 {
         return 0;
 }
 #endif
 
 #ifdef GTK2_SUPPORT
-int symbol__gtk_annotate(struct symbol *sym, struct map *map, int evidx,
+int symbol__gtk_annotate(struct symbol *sym, struct map *map,
+                         struct perf_evsel *evsel,
                          struct hist_browser_timer *hbt);
 
-static inline int hist_entry__gtk_annotate(struct hist_entry *he, int evidx,
+static inline int hist_entry__gtk_annotate(struct hist_entry *he,
+                                           struct perf_evsel *evsel,
                                            struct hist_browser_timer *hbt)
 {
-        return symbol__gtk_annotate(he->ms.sym, he->ms.map, evidx, hbt);
+        return symbol__gtk_annotate(he->ms.sym, he->ms.map, evsel, hbt);
 }
 
 void perf_gtk__show_annotations(void);
 #else
 static inline int hist_entry__gtk_annotate(struct hist_entry *he __maybe_unused,
-                                           int evidx __maybe_unused,
-                                           struct hist_browser_timer *hbt
-                                           __maybe_unused)
+                                struct perf_evsel *evsel __maybe_unused,
+                                struct hist_browser_timer *hbt __maybe_unused)
 {
         return 0;
 }
diff --git a/tools/perf/util/cpumap.c b/tools/perf/util/cpumap.c
index f817046e22b1..beb8cf9f9976 100644
--- a/tools/perf/util/cpumap.c
+++ b/tools/perf/util/cpumap.c
@@ -4,6 +4,7 @@
 #include "cpumap.h"
 #include <assert.h>
 #include <stdio.h>
+#include <stdlib.h>
 
 static struct cpu_map *cpu_map__default_new(void)
 {
@@ -219,7 +220,7 @@ int cpu_map__get_socket(struct cpu_map *map, int idx)
         if (!mnt)
                 return -1;
 
-        sprintf(path,
+        snprintf(path, PATH_MAX,
                 "%s/devices/system/cpu/cpu%d/topology/physical_package_id",
                 mnt, cpu);
 
@@ -231,27 +232,88 @@ int cpu_map__get_socket(struct cpu_map *map, int idx)
         return ret == 1 ? cpu : -1;
 }
 
-int cpu_map__build_socket_map(struct cpu_map *cpus, struct cpu_map **sockp)
+static int cmp_ids(const void *a, const void *b)
 {
-        struct cpu_map *sock;
+        return *(int *)a - *(int *)b;
+}
+
+static int cpu_map__build_map(struct cpu_map *cpus, struct cpu_map **res,
+                              int (*f)(struct cpu_map *map, int cpu))
+{
+        struct cpu_map *c;
         int nr = cpus->nr;
         int cpu, s1, s2;
 
-        sock = calloc(1, sizeof(*sock) + nr * sizeof(int));
-        if (!sock)
+        /* allocate as much as possible */
+        c = calloc(1, sizeof(*c) + nr * sizeof(int));
+        if (!c)
                 return -1;
 
         for (cpu = 0; cpu < nr; cpu++) {
-                s1 = cpu_map__get_socket(cpus, cpu);
-                for (s2 = 0; s2 < sock->nr; s2++) {
-                        if (s1 == sock->map[s2])
+                s1 = f(cpus, cpu);
+                for (s2 = 0; s2 < c->nr; s2++) {
+                        if (s1 == c->map[s2])
                                 break;
                 }
-                if (s2 == sock->nr) {
-                        sock->map[sock->nr] = s1;
-                        sock->nr++;
+                if (s2 == c->nr) {
+                        c->map[c->nr] = s1;
+                        c->nr++;
                 }
         }
-        *sockp = sock;
+        /* ensure we process id in increasing order */
+        qsort(c->map, c->nr, sizeof(int), cmp_ids);
+
+        *res = c;
         return 0;
 }
+
+int cpu_map__get_core(struct cpu_map *map, int idx)
+{
+        FILE *fp;
+        const char *mnt;
+        char path[PATH_MAX];
+        int cpu, ret, s;
+
+        if (idx > map->nr)
+                return -1;
+
+        cpu = map->map[idx];
+
+        mnt = sysfs_find_mountpoint();
+        if (!mnt)
+                return -1;
+
+        snprintf(path, PATH_MAX,
+                "%s/devices/system/cpu/cpu%d/topology/core_id",
+                mnt, cpu);
+
+        fp = fopen(path, "r");
+        if (!fp)
+                return -1;
+        ret = fscanf(fp, "%d", &cpu);
+        fclose(fp);
+        if (ret != 1)
+                return -1;
+
+        s = cpu_map__get_socket(map, idx);
+        if (s == -1)
+                return -1;
+
+        /*
+         * encode socket in upper 16 bits
+         * core_id is relative to socket, and
+         * we need a global id. So we combine
+         * socket+ core id
+         */
+        return (s << 16) | (cpu & 0xffff);
+}
+
+int cpu_map__build_socket_map(struct cpu_map *cpus, struct cpu_map **sockp)
+{
+        return cpu_map__build_map(cpus, sockp, cpu_map__get_socket);
+}
+
+int cpu_map__build_core_map(struct cpu_map *cpus, struct cpu_map **corep)
+{
+        return cpu_map__build_map(cpus, corep, cpu_map__get_core);
+}
diff --git a/tools/perf/util/cpumap.h b/tools/perf/util/cpumap.h
index 161b00756a12..9bed02e5fb3d 100644
--- a/tools/perf/util/cpumap.h
+++ b/tools/perf/util/cpumap.h
@@ -15,7 +15,9 @@ void cpu_map__delete(struct cpu_map *map);
 struct cpu_map *cpu_map__read(FILE *file);
 size_t cpu_map__fprintf(struct cpu_map *map, FILE *fp);
 int cpu_map__get_socket(struct cpu_map *map, int idx);
+int cpu_map__get_core(struct cpu_map *map, int idx);
 int cpu_map__build_socket_map(struct cpu_map *cpus, struct cpu_map **sockp);
+int cpu_map__build_core_map(struct cpu_map *cpus, struct cpu_map **corep);
 
 static inline int cpu_map__socket(struct cpu_map *sock, int s)
 {
@@ -24,6 +26,16 @@ static inline int cpu_map__socket(struct cpu_map *sock, int s)
         return sock->map[s];
 }
 
+static inline int cpu_map__id_to_socket(int id)
+{
+        return id >> 16;
+}
+
+static inline int cpu_map__id_to_cpu(int id)
+{
+        return id & 0xffff;
+}
+
 static inline int cpu_map__nr(const struct cpu_map *map)
 {
         return map ? map->nr : 1;
diff --git a/tools/perf/util/debugfs.h b/tools/perf/util/debugfs.h
deleted file mode 100644
index 68f3e87ec57f..000000000000
--- a/tools/perf/util/debugfs.h
+++ /dev/null
@@ -1,12 +0,0 @@
-#ifndef __DEBUGFS_H__
-#define __DEBUGFS_H__
-
-const char *debugfs_find_mountpoint(void);
-int debugfs_valid_mountpoint(const char *debugfs);
-char *debugfs_mount(const char *mountpoint);
-void debugfs_set_path(const char *mountpoint);
-
-extern char debugfs_mountpoint[];
-extern char tracing_events_path[];
-
-#endif /* __DEBUGFS_H__ */
diff --git a/tools/perf/util/event.h b/tools/perf/util/event.h
index 0d573ff4771a..181389535c0c 100644
--- a/tools/perf/util/event.h
+++ b/tools/perf/util/event.h
@@ -88,8 +88,10 @@ struct perf_sample {
         u64 id;
         u64 stream_id;
         u64 period;
+        u64 weight;
         u32 cpu;
         u32 raw_size;
+        u64 data_src;
         void *raw_data;
         struct ip_callchain *callchain;
         struct branch_stack *branch_stack;
@@ -97,6 +99,13 @@ struct perf_sample {
         struct stack_dump user_stack;
 };
 
+#define PERF_MEM_DATA_SRC_NONE \
+        (PERF_MEM_S(OP, NA) |\
+         PERF_MEM_S(LVL, NA) |\
+         PERF_MEM_S(SNOOP, NA) |\
+         PERF_MEM_S(LOCK, NA) |\
+         PERF_MEM_S(TLB, NA))
+
 struct build_id_event {
         struct perf_event_header header;
         pid_t                    pid;
diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
index c8be0fbc5145..f7c727801aab 100644
--- a/tools/perf/util/evlist.c
+++ b/tools/perf/util/evlist.c
@@ -7,7 +7,7 @@
  * Released under the GPL v2. (and only v2, not any later version)
  */
 #include "util.h"
-#include "debugfs.h"
+#include <lk/debugfs.h>
 #include <poll.h>
 #include "cpumap.h"
 #include "thread_map.h"
@@ -38,13 +38,12 @@ void perf_evlist__init(struct perf_evlist *evlist, struct cpu_map *cpus,
         evlist->workload.pid = -1;
 }
 
-struct perf_evlist *perf_evlist__new(struct cpu_map *cpus,
-                                     struct thread_map *threads)
+struct perf_evlist *perf_evlist__new(void)
 {
         struct perf_evlist *evlist = zalloc(sizeof(*evlist));
 
         if (evlist != NULL)
-                perf_evlist__init(evlist, cpus, threads);
+                perf_evlist__init(evlist, NULL, NULL);
 
         return evlist;
 }
@@ -228,12 +227,14 @@ void perf_evlist__disable(struct perf_evlist *evlist)
 {
         int cpu, thread;
         struct perf_evsel *pos;
+        int nr_cpus = cpu_map__nr(evlist->cpus);
+        int nr_threads = thread_map__nr(evlist->threads);
 
-        for (cpu = 0; cpu < evlist->cpus->nr; cpu++) {
+        for (cpu = 0; cpu < nr_cpus; cpu++) {
                 list_for_each_entry(pos, &evlist->entries, node) {
                         if (!perf_evsel__is_group_leader(pos))
                                 continue;
-                        for (thread = 0; thread < evlist->threads->nr; thread++)
+                        for (thread = 0; thread < nr_threads; thread++)
                                 ioctl(FD(pos, cpu, thread),
                                       PERF_EVENT_IOC_DISABLE, 0);
                 }
@@ -244,12 +245,14 @@ void perf_evlist__enable(struct perf_evlist *evlist)
 {
         int cpu, thread;
         struct perf_evsel *pos;
+        int nr_cpus = cpu_map__nr(evlist->cpus);
+        int nr_threads = thread_map__nr(evlist->threads);
 
-        for (cpu = 0; cpu < cpu_map__nr(evlist->cpus); cpu++) {
+        for (cpu = 0; cpu < nr_cpus; cpu++) {
                 list_for_each_entry(pos, &evlist->entries, node) {
                         if (!perf_evsel__is_group_leader(pos))
                                 continue;
-                        for (thread = 0; thread < evlist->threads->nr; thread++)
+                        for (thread = 0; thread < nr_threads; thread++)
                                 ioctl(FD(pos, cpu, thread),
                                       PERF_EVENT_IOC_ENABLE, 0);
                 }
@@ -258,7 +261,9 @@ void perf_evlist__enable(struct perf_evlist *evlist)
 
 static int perf_evlist__alloc_pollfd(struct perf_evlist *evlist)
 {
-        int nfds = cpu_map__nr(evlist->cpus) * evlist->threads->nr * evlist->nr_entries;
+        int nr_cpus = cpu_map__nr(evlist->cpus);
+        int nr_threads = thread_map__nr(evlist->threads);
+        int nfds = nr_cpus * nr_threads * evlist->nr_entries;
         evlist->pollfd = malloc(sizeof(struct pollfd) * nfds);
         return evlist->pollfd != NULL ? 0 : -ENOMEM;
 }
@@ -417,7 +422,7 @@ static int perf_evlist__alloc_mmap(struct perf_evlist *evlist)
 {
         evlist->nr_mmaps = cpu_map__nr(evlist->cpus);
         if (cpu_map__all(evlist->cpus))
-                evlist->nr_mmaps = evlist->threads->nr;
+                evlist->nr_mmaps = thread_map__nr(evlist->threads);
         evlist->mmap = zalloc(evlist->nr_mmaps * sizeof(struct perf_mmap));
         return evlist->mmap != NULL ? 0 : -ENOMEM;
 }
@@ -442,11 +447,13 @@ static int perf_evlist__mmap_per_cpu(struct perf_evlist *evlist, int prot, int m
 {
         struct perf_evsel *evsel;
         int cpu, thread;
+        int nr_cpus = cpu_map__nr(evlist->cpus);
+        int nr_threads = thread_map__nr(evlist->threads);
 
-        for (cpu = 0; cpu < evlist->cpus->nr; cpu++) {
+        for (cpu = 0; cpu < nr_cpus; cpu++) {
                 int output = -1;
 
-                for (thread = 0; thread < evlist->threads->nr; thread++) {
+                for (thread = 0; thread < nr_threads; thread++) {
                         list_for_each_entry(evsel, &evlist->entries, node) {
                                 int fd = FD(evsel, cpu, thread);
 
@@ -470,7 +477,7 @@ static int perf_evlist__mmap_per_cpu(struct perf_evlist *evlist, int prot, int m
         return 0;
 
 out_unmap:
-        for (cpu = 0; cpu < evlist->cpus->nr; cpu++) {
+        for (cpu = 0; cpu < nr_cpus; cpu++) {
                 if (evlist->mmap[cpu].base != NULL) {
                         munmap(evlist->mmap[cpu].base, evlist->mmap_len);
                         evlist->mmap[cpu].base = NULL;
@@ -483,8 +490,9 @@ static int perf_evlist__mmap_per_thread(struct perf_evlist *evlist, int prot, in
 {
         struct perf_evsel *evsel;
         int thread;
+        int nr_threads = thread_map__nr(evlist->threads);
 
-        for (thread = 0; thread < evlist->threads->nr; thread++) {
+        for (thread = 0; thread < nr_threads; thread++) {
                 int output = -1;
 
                 list_for_each_entry(evsel, &evlist->entries, node) {
@@ -509,7 +517,7 @@ static int perf_evlist__mmap_per_thread(struct perf_evlist *evlist, int prot, in
         return 0;
 
 out_unmap:
-        for (thread = 0; thread < evlist->threads->nr; thread++) {
+        for (thread = 0; thread < nr_threads; thread++) {
                 if (evlist->mmap[thread].base != NULL) {
                         munmap(evlist->mmap[thread].base, evlist->mmap_len);
                         evlist->mmap[thread].base = NULL;
@@ -610,7 +618,7 @@ int perf_evlist__apply_filters(struct perf_evlist *evlist)
         struct perf_evsel *evsel;
         int err = 0;
         const int ncpus = cpu_map__nr(evlist->cpus),
-                  nthreads = evlist->threads->nr;
+                  nthreads = thread_map__nr(evlist->threads);
 
         list_for_each_entry(evsel, &evlist->entries, node) {
                 if (evsel->filter == NULL)
@@ -629,7 +637,7 @@ int perf_evlist__set_filter(struct perf_evlist *evlist, const char *filter)
         struct perf_evsel *evsel;
         int err = 0;
         const int ncpus = cpu_map__nr(evlist->cpus),
-                  nthreads = evlist->threads->nr;
+                  nthreads = thread_map__nr(evlist->threads);
 
         list_for_each_entry(evsel, &evlist->entries, node) {
                 err = perf_evsel__set_filter(evsel, ncpus, nthreads, filter);
@@ -712,10 +720,20 @@ void perf_evlist__set_selected(struct perf_evlist *evlist,
         evlist->selected = evsel;
 }
 
+void perf_evlist__close(struct perf_evlist *evlist)
+{
+        struct perf_evsel *evsel;
+        int ncpus = cpu_map__nr(evlist->cpus);
+        int nthreads = thread_map__nr(evlist->threads);
+
+        list_for_each_entry_reverse(evsel, &evlist->entries, node)
+                perf_evsel__close(evsel, ncpus, nthreads);
+}
+
 int perf_evlist__open(struct perf_evlist *evlist)
 {
         struct perf_evsel *evsel;
-        int err, ncpus, nthreads;
+        int err;
 
         list_for_each_entry(evsel, &evlist->entries, node) {
                 err = perf_evsel__open(evsel, evlist->cpus, evlist->threads);
@@ -725,19 +743,15 @@ int perf_evlist__open(struct perf_evlist *evlist)
 
         return 0;
 out_err:
-        ncpus = evlist->cpus ? evlist->cpus->nr : 1;
-        nthreads = evlist->threads ? evlist->threads->nr : 1;
-
-        list_for_each_entry_reverse(evsel, &evlist->entries, node)
-                perf_evsel__close(evsel, ncpus, nthreads);
-
+        perf_evlist__close(evlist);
         errno = -err;
         return err;
 }
 
 int perf_evlist__prepare_workload(struct perf_evlist *evlist,
-                                  struct perf_record_opts *opts,
-                                  const char *argv[])
+                                  struct perf_target *target,
+                                  const char *argv[], bool pipe_output,
+                                  bool want_signal)
 {
         int child_ready_pipe[2], go_pipe[2];
         char bf;
@@ -759,7 +773,7 @@ int perf_evlist__prepare_workload(struct perf_evlist *evlist,
         }
 
         if (!evlist->workload.pid) {
-                if (opts->pipe_output)
+                if (pipe_output)
                         dup2(2, 1);
 
                 close(child_ready_pipe[0]);
@@ -787,11 +801,12 @@ int perf_evlist__prepare_workload(struct perf_evlist *evlist,
                 execvp(argv[0], (char **)argv);
 
                 perror(argv[0]);
-                kill(getppid(), SIGUSR1);
+                if (want_signal)
+                        kill(getppid(), SIGUSR1);
                 exit(-1);
         }
 
-        if (perf_target__none(&opts->target))
+        if (perf_target__none(target))
                 evlist->threads->map[0] = evlist->workload.pid;
 
         close(child_ready_pipe[1]);
diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
index 2dd07bd60b4f..0583d36252be 100644
--- a/tools/perf/util/evlist.h
+++ b/tools/perf/util/evlist.h
@@ -49,8 +49,7 @@ struct perf_evsel_str_handler {
         void       *handler;
 };
 
-struct perf_evlist *perf_evlist__new(struct cpu_map *cpus,
-                                     struct thread_map *threads);
+struct perf_evlist *perf_evlist__new(void);
 void perf_evlist__init(struct perf_evlist *evlist, struct cpu_map *cpus,
                        struct thread_map *threads);
 void perf_evlist__exit(struct perf_evlist *evlist);
@@ -82,13 +81,15 @@ struct perf_evsel *perf_evlist__id2evsel(struct perf_evlist *evlist, u64 id);
 union perf_event *perf_evlist__mmap_read(struct perf_evlist *self, int idx);
 
 int perf_evlist__open(struct perf_evlist *evlist);
+void perf_evlist__close(struct perf_evlist *evlist);
 
 void perf_evlist__config(struct perf_evlist *evlist,
                          struct perf_record_opts *opts);
 
 int perf_evlist__prepare_workload(struct perf_evlist *evlist,
-                                  struct perf_record_opts *opts,
-                                  const char *argv[]);
+                                  struct perf_target *target,
+                                  const char *argv[], bool pipe_output,
+                                  bool want_signal);
 int perf_evlist__start_workload(struct perf_evlist *evlist);
 
 int perf_evlist__mmap(struct perf_evlist *evlist, unsigned int pages,
diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c
index 9c82f98f26de..07b1a3ad3e24 100644
--- a/tools/perf/util/evsel.c
+++ b/tools/perf/util/evsel.c
@@ -10,7 +10,7 @@
 #include <byteswap.h>
 #include <linux/bitops.h>
 #include "asm/bug.h"
-#include "debugfs.h"
+#include <lk/debugfs.h>
 #include "event-parse.h"
 #include "evsel.h"
 #include "evlist.h"
@@ -554,6 +554,9 @@ void perf_evsel__config(struct perf_evsel *evsel,
                 perf_evsel__set_sample_bit(evsel, CPU);
         }
 
+        if (opts->sample_address)
+                attr->sample_type       |= PERF_SAMPLE_DATA_SRC;
+
         if (opts->no_delay) {
                 attr->watermark = 0;
                 attr->wakeup_events = 1;
@@ -563,6 +566,9 @@ void perf_evsel__config(struct perf_evsel *evsel,
                 attr->branch_sample_type = opts->branch_stack;
         }
 
+        if (opts->sample_weight)
+                attr->sample_type       |= PERF_SAMPLE_WEIGHT;
+
         attr->mmap = track;
         attr->comm = track;
 
@@ -633,6 +639,12 @@ int perf_evsel__alloc_id(struct perf_evsel *evsel, int ncpus, int nthreads)
         return 0;
 }
 
+void perf_evsel__reset_counts(struct perf_evsel *evsel, int ncpus)
+{
+        memset(evsel->counts, 0, (sizeof(*evsel->counts) +
+                                 (ncpus * sizeof(struct perf_counts_values))));
+}
+
 int perf_evsel__alloc_counts(struct perf_evsel *evsel, int ncpus)
 {
         evsel->counts = zalloc((sizeof(*evsel->counts) +
@@ -673,9 +685,8 @@ void perf_evsel__free_counts(struct perf_evsel *evsel)
 void perf_evsel__exit(struct perf_evsel *evsel)
 {
         assert(list_empty(&evsel->node));
-        xyarray__delete(evsel->fd);
-        xyarray__delete(evsel->sample_id);
-        free(evsel->id);
+        perf_evsel__free_fd(evsel);
+        perf_evsel__free_id(evsel);
 }
 
 void perf_evsel__delete(struct perf_evsel *evsel)
@@ -1012,6 +1023,7 @@ int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event,
         data->cpu = data->pid = data->tid = -1;
         data->stream_id = data->id = data->time = -1ULL;
         data->period = 1;
+        data->weight = 0;
 
         if (event->header.type != PERF_RECORD_SAMPLE) {
                 if (!evsel->attr.sample_id_all)
@@ -1162,6 +1174,18 @@ int perf_evsel__parse_sample(struct perf_evsel *evsel, union perf_event *event,
                 }
         }
 
+        data->weight = 0;
+        if (type & PERF_SAMPLE_WEIGHT) {
+                data->weight = *array;
+                array++;
+        }
+
+        data->data_src = PERF_MEM_DATA_SRC_NONE;
+        if (type & PERF_SAMPLE_DATA_SRC) {
+                data->data_src = *array;
+                array++;
+        }
+
         return 0;
 }
 
diff --git a/tools/perf/util/evsel.h b/tools/perf/util/evsel.h
index 52021c3087df..3f156ccc1acb 100644
--- a/tools/perf/util/evsel.h
+++ b/tools/perf/util/evsel.h
@@ -9,6 +9,7 @@
 #include "xyarray.h"
 #include "cgroup.h"
 #include "hist.h"
+#include "symbol.h"
  
 struct perf_counts_values {
         union {
@@ -120,6 +121,7 @@ int perf_evsel__group_desc(struct perf_evsel *evsel, char *buf, size_t size);
 int perf_evsel__alloc_fd(struct perf_evsel *evsel, int ncpus, int nthreads);
 int perf_evsel__alloc_id(struct perf_evsel *evsel, int ncpus, int nthreads);
 int perf_evsel__alloc_counts(struct perf_evsel *evsel, int ncpus);
+void perf_evsel__reset_counts(struct perf_evsel *evsel, int ncpus);
 void perf_evsel__free_fd(struct perf_evsel *evsel);
 void perf_evsel__free_id(struct perf_evsel *evsel);
 void perf_evsel__free_counts(struct perf_evsel *evsel);
@@ -246,11 +248,34 @@ static inline struct perf_evsel *perf_evsel__next(struct perf_evsel *evsel)
         return list_entry(evsel->node.next, struct perf_evsel, node);
 }
 
+/**
+ * perf_evsel__is_group_leader - Return whether given evsel is a leader event
+ *
+ * @evsel - evsel selector to be tested
+ *
+ * Return %true if @evsel is a group leader or a stand-alone event
+ */
 static inline bool perf_evsel__is_group_leader(const struct perf_evsel *evsel)
 {
         return evsel->leader == evsel;
 }
 
+/**
+ * perf_evsel__is_group_event - Return whether given evsel is a group event
+ *
+ * @evsel - evsel selector to be tested
+ *
+ * Return %true iff event group view is enabled and @evsel is a actual group
+ * leader which has other members in the group
+ */
+static inline bool perf_evsel__is_group_event(struct perf_evsel *evsel)
+{
+        if (!symbol_conf.event_group)
+                return false;
+
+        return perf_evsel__is_group_leader(evsel) && evsel->nr_members > 1;
+}
+
 struct perf_attr_details {
         bool freq;
         bool verbose;
diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c
index f4bfd79ef6a7..326068a593a5 100644
--- a/tools/perf/util/header.c
+++ b/tools/perf/util/header.c
@@ -1,5 +1,3 @@
-#define _FILE_OFFSET_BITS 64
-
 #include "util.h"
 #include <sys/types.h>
 #include <byteswap.h>
@@ -1672,8 +1670,8 @@ static int process_tracing_data(struct perf_file_section *section __maybe_unused
                                 struct perf_header *ph __maybe_unused,
                                 int fd, void *data)
 {
-        trace_report(fd, data, false);
-        return 0;
+        ssize_t ret = trace_report(fd, data, false);
+        return ret < 0 ? -1 : 0;
 }
 
 static int process_build_id(struct perf_file_section *section,
@@ -2752,6 +2750,11 @@ static int perf_evsel__prepare_tracepoint_event(struct perf_evsel *evsel,
         if (evsel->tp_format)
                 return 0;
 
+        if (pevent == NULL) {
+                pr_debug("broken or missing trace data\n");
+                return -1;
+        }
+
         event = pevent_find_event(pevent, evsel->attr.config);
         if (event == NULL)
                 return -1;
@@ -2789,7 +2792,7 @@ int perf_session__read_header(struct perf_session *session, int fd)
         u64                     f_id;
         int nr_attrs, nr_ids, i, j;
 
-        session->evlist = perf_evlist__new(NULL, NULL);
+        session->evlist = perf_evlist__new();
         if (session->evlist == NULL)
                 return -ENOMEM;
 
@@ -2940,7 +2943,7 @@ int perf_event__process_attr(union perf_event *event,
         struct perf_evlist *evlist = *pevlist;
 
         if (evlist == NULL) {
-                *pevlist = evlist = perf_evlist__new(NULL, NULL);
+                *pevlist = evlist = perf_evlist__new();
                 if (evlist == NULL)
                         return -ENOMEM;
         }
diff --git a/tools/perf/util/hist.c b/tools/perf/util/hist.c
index f855941bebea..6b32721f829a 100644
--- a/tools/perf/util/hist.c
+++ b/tools/perf/util/hist.c
@@ -67,12 +67,16 @@ static void hists__set_unres_dso_col_len(struct hists *hists, int dso)
 void hists__calc_col_len(struct hists *hists, struct hist_entry *h)
 {
         const unsigned int unresolved_col_width = BITS_PER_LONG / 4;
+        int symlen;
         u16 len;
 
         if (h->ms.sym)
                 hists__new_col_len(hists, HISTC_SYMBOL, h->ms.sym->namelen + 4);
-        else
+        else {
+                symlen = unresolved_col_width + 4 + 2;
+                hists__new_col_len(hists, HISTC_SYMBOL, symlen);
                 hists__set_unres_dso_col_len(hists, HISTC_DSO);
+        }
 
         len = thread__comm_len(h->thread);
         if (hists__new_col_len(hists, HISTC_COMM, len))
@@ -87,7 +91,6 @@ void hists__calc_col_len(struct hists *hists, struct hist_entry *h)
                 hists__new_col_len(hists, HISTC_PARENT, h->parent->namelen);
 
         if (h->branch_info) {
-                int symlen;
                 /*
                  * +4 accounts for '[x] ' priv level info
                  * +2 account of 0x prefix on raw addresses
@@ -116,6 +119,42 @@ void hists__calc_col_len(struct hists *hists, struct hist_entry *h)
                         hists__set_unres_dso_col_len(hists, HISTC_DSO_TO);
                 }
         }
+
+        if (h->mem_info) {
+                /*
+                 * +4 accounts for '[x] ' priv level info
+                 * +2 account of 0x prefix on raw addresses
+                 */
+                if (h->mem_info->daddr.sym) {
+                        symlen = (int)h->mem_info->daddr.sym->namelen + 4
+                               + unresolved_col_width + 2;
+                        hists__new_col_len(hists, HISTC_MEM_DADDR_SYMBOL,
+                                           symlen);
+                } else {
+                        symlen = unresolved_col_width + 4 + 2;
+                        hists__new_col_len(hists, HISTC_MEM_DADDR_SYMBOL,
+                                           symlen);
+                }
+                if (h->mem_info->daddr.map) {
+                        symlen = dso__name_len(h->mem_info->daddr.map->dso);
+                        hists__new_col_len(hists, HISTC_MEM_DADDR_DSO,
+                                           symlen);
+                } else {
+                        symlen = unresolved_col_width + 4 + 2;
+                        hists__set_unres_dso_col_len(hists, HISTC_MEM_DADDR_DSO);
+                }
+        } else {
+                symlen = unresolved_col_width + 4 + 2;
+                hists__new_col_len(hists, HISTC_MEM_DADDR_SYMBOL, symlen);
+                hists__set_unres_dso_col_len(hists, HISTC_MEM_DADDR_DSO);
+        }
+
+        hists__new_col_len(hists, HISTC_MEM_LOCKED, 6);
+        hists__new_col_len(hists, HISTC_MEM_TLB, 22);
+        hists__new_col_len(hists, HISTC_MEM_SNOOP, 12);
+        hists__new_col_len(hists, HISTC_MEM_LVL, 21 + 3);
+        hists__new_col_len(hists, HISTC_LOCAL_WEIGHT, 12);
+        hists__new_col_len(hists, HISTC_GLOBAL_WEIGHT, 12);
 }
 
 void hists__output_recalc_col_len(struct hists *hists, int max_rows)
@@ -155,9 +194,12 @@ static void hist_entry__add_cpumode_period(struct hist_entry *he,
         }
 }
 
-static void he_stat__add_period(struct he_stat *he_stat, u64 period)
+static void he_stat__add_period(struct he_stat *he_stat, u64 period,
+                                u64 weight)
 {
+
         he_stat->period         += period;
+        he_stat->weight         += weight;
         he_stat->nr_events      += 1;
 }
 
@@ -169,12 +211,14 @@ static void he_stat__add_stat(struct he_stat *dest, struct he_stat *src)
         dest->period_guest_sys  += src->period_guest_sys;
         dest->period_guest_us   += src->period_guest_us;
         dest->nr_events         += src->nr_events;
+        dest->weight            += src->weight;
 }
 
 static void hist_entry__decay(struct hist_entry *he)
 {
         he->stat.period = (he->stat.period * 7) / 8;
         he->stat.nr_events = (he->stat.nr_events * 7) / 8;
+        /* XXX need decay for weight too? */
 }
 
 static bool hists__decay_entry(struct hists *hists, struct hist_entry *he)
@@ -239,7 +283,7 @@ void hists__decay_entries_threaded(struct hists *hists,
 static struct hist_entry *hist_entry__new(struct hist_entry *template)
 {
         size_t callchain_size = symbol_conf.use_callchain ? sizeof(struct callchain_root) : 0;
-        struct hist_entry *he = malloc(sizeof(*he) + callchain_size);
+        struct hist_entry *he = zalloc(sizeof(*he) + callchain_size);
 
         if (he != NULL) {
                 *he = *template;
@@ -254,6 +298,13 @@ static struct hist_entry *hist_entry__new(struct hist_entry *template)
                                 he->branch_info->to.map->referenced = true;
                 }
 
+                if (he->mem_info) {
+                        if (he->mem_info->iaddr.map)
+                                he->mem_info->iaddr.map->referenced = true;
+                        if (he->mem_info->daddr.map)
+                                he->mem_info->daddr.map->referenced = true;
+                }
+
                 if (symbol_conf.use_callchain)
                         callchain_init(he->callchain);
 
@@ -282,7 +333,8 @@ static u8 symbol__parent_filter(const struct symbol *parent)
 static struct hist_entry *add_hist_entry(struct hists *hists,
                                       struct hist_entry *entry,
                                       struct addr_location *al,
-                                      u64 period)
+                                      u64 period,
+                                      u64 weight)
 {
         struct rb_node **p;
         struct rb_node *parent = NULL;
@@ -306,7 +358,7 @@ static struct hist_entry *add_hist_entry(struct hists *hists,
                 cmp = hist_entry__cmp(he, entry);
 
                 if (!cmp) {
-                        he_stat__add_period(&he->stat, period);
+                        he_stat__add_period(&he->stat, period, weight);
 
                         /* If the map of an existing hist_entry has
                          * become out-of-date due to an exec() or
@@ -341,11 +393,42 @@ out_unlock:
         return he;
 }
 
+struct hist_entry *__hists__add_mem_entry(struct hists *self,
+                                          struct addr_location *al,
+                                          struct symbol *sym_parent,
+                                          struct mem_info *mi,
+                                          u64 period,
+                                          u64 weight)
+{
+        struct hist_entry entry = {
+                .thread = al->thread,
+                .ms = {
+                        .map    = al->map,
+                        .sym    = al->sym,
+                },
+                .stat = {
+                        .period = period,
+                        .weight = weight,
+                        .nr_events = 1,
+                },
+                .cpu    = al->cpu,
+                .ip     = al->addr,
+                .level  = al->level,
+                .parent = sym_parent,
+                .filtered = symbol__parent_filter(sym_parent),
+                .hists = self,
+                .mem_info = mi,
+                .branch_info = NULL,
+        };
+        return add_hist_entry(self, &entry, al, period, weight);
+}
+
 struct hist_entry *__hists__add_branch_entry(struct hists *self,
                                              struct addr_location *al,
                                              struct symbol *sym_parent,
                                              struct branch_info *bi,
-                                             u64 period)
+                                             u64 period,
+                                             u64 weight)
 {
         struct hist_entry entry = {
                 .thread = al->thread,
@@ -359,19 +442,22 @@ struct hist_entry *__hists__add_branch_entry(struct hists *self,
                 .stat = {
                         .period = period,
                         .nr_events = 1,
+                        .weight = weight,
                 },
                 .parent = sym_parent,
                 .filtered = symbol__parent_filter(sym_parent),
                 .branch_info = bi,
                 .hists  = self,
+                .mem_info = NULL,
         };
 
-        return add_hist_entry(self, &entry, al, period);
+        return add_hist_entry(self, &entry, al, period, weight);
 }
 
 struct hist_entry *__hists__add_entry(struct hists *self,
                                       struct addr_location *al,
-                                      struct symbol *sym_parent, u64 period)
+                                      struct symbol *sym_parent, u64 period,
+                                      u64 weight)
 {
         struct hist_entry entry = {
                 .thread = al->thread,
@@ -385,13 +471,16 @@ struct hist_entry *__hists__add_entry(struct hists *self,
                 .stat = {
                         .period = period,
                         .nr_events = 1,
+                        .weight = weight,
                 },
                 .parent = sym_parent,
                 .filtered = symbol__parent_filter(sym_parent),
                 .hists  = self,
+                .branch_info = NULL,
+                .mem_info = NULL,
         };
 
-        return add_hist_entry(self, &entry, al, period);
+        return add_hist_entry(self, &entry, al, period, weight);
 }
 
 int64_t
@@ -431,6 +520,7 @@ hist_entry__collapse(struct hist_entry *left, struct hist_entry *right)
 void hist_entry__free(struct hist_entry *he)
 {
         free(he->branch_info);
+        free(he->mem_info);
         free(he);
 }
 
diff --git a/tools/perf/util/hist.h b/tools/perf/util/hist.h
index 226a4ae2f936..14c2fe20aa62 100644
--- a/tools/perf/util/hist.h
+++ b/tools/perf/util/hist.h
@@ -49,6 +49,14 @@ enum hist_column {
         HISTC_DSO_FROM,
         HISTC_DSO_TO,
         HISTC_SRCLINE,
+        HISTC_LOCAL_WEIGHT,
+        HISTC_GLOBAL_WEIGHT,
+        HISTC_MEM_DADDR_SYMBOL,
+        HISTC_MEM_DADDR_DSO,
+        HISTC_MEM_LOCKED,
+        HISTC_MEM_TLB,
+        HISTC_MEM_LVL,
+        HISTC_MEM_SNOOP,
         HISTC_NR_COLS, /* Last entry */
 };
 
@@ -73,7 +81,8 @@ struct hists {
 
 struct hist_entry *__hists__add_entry(struct hists *self,
                                       struct addr_location *al,
-                                      struct symbol *parent, u64 period);
+                                      struct symbol *parent, u64 period,
+                                      u64 weight);
 int64_t hist_entry__cmp(struct hist_entry *left, struct hist_entry *right);
 int64_t hist_entry__collapse(struct hist_entry *left, struct hist_entry *right);
 int hist_entry__sort_snprintf(struct hist_entry *self, char *bf, size_t size,
@@ -84,7 +93,15 @@ struct hist_entry *__hists__add_branch_entry(struct hists *self,
                                              struct addr_location *al,
                                              struct symbol *sym_parent,
                                              struct branch_info *bi,
-                                             u64 period);
+                                             u64 period,
+                                             u64 weight);
+
+struct hist_entry *__hists__add_mem_entry(struct hists *self,
+                                          struct addr_location *al,
+                                          struct symbol *sym_parent,
+                                          struct mem_info *mi,
+                                          u64 period,
+                                          u64 weight);
 
 void hists__output_resort(struct hists *self);
 void hists__output_resort_threaded(struct hists *hists);
@@ -175,9 +192,9 @@ struct hist_browser_timer {
         int refresh;
 };
 
-#ifdef NEWT_SUPPORT
+#ifdef SLANG_SUPPORT
 #include "../ui/keysyms.h"
-int hist_entry__tui_annotate(struct hist_entry *he, int evidx,
+int hist_entry__tui_annotate(struct hist_entry *he, struct perf_evsel *evsel,
                              struct hist_browser_timer *hbt);
 
 int perf_evlist__tui_browse_hists(struct perf_evlist *evlist, const char *help,
@@ -196,7 +213,8 @@ int perf_evlist__tui_browse_hists(struct perf_evlist *evlist __maybe_unused,
 
 static inline int hist_entry__tui_annotate(struct hist_entry *self
                                            __maybe_unused,
-                                           int evidx __maybe_unused,
+                                           struct perf_evsel *evsel
+                                           __maybe_unused,
                                            struct hist_browser_timer *hbt
                                            __maybe_unused)
 {
diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c
index efdb38e65a92..b2ecad6ec46b 100644
--- a/tools/perf/util/machine.c
+++ b/tools/perf/util/machine.c
@@ -955,6 +955,7 @@ int machine__process_mmap_event(struct machine *machine, union perf_event *event
         u8 cpumode = event->header.misc & PERF_RECORD_MISC_CPUMODE_MASK;
         struct thread *thread;
         struct map *map;
+        enum map_type type;
         int ret = 0;
 
         if (dump_trace)
@@ -971,10 +972,17 @@ int machine__process_mmap_event(struct machine *machine, union perf_event *event
         thread = machine__findnew_thread(machine, event->mmap.pid);
         if (thread == NULL)
                 goto out_problem;
+
+        if (event->header.misc & PERF_RECORD_MISC_MMAP_DATA)
+                type = MAP__VARIABLE;
+        else
+                type = MAP__FUNCTION;
+
         map = map__new(&machine->user_dsos, event->mmap.start,
                         event->mmap.len, event->mmap.pgoff,
                         event->mmap.pid, event->mmap.filename,
-                        MAP__FUNCTION);
+                        type);
+
         if (map == NULL)
                 goto out_problem;
 
@@ -1003,6 +1011,17 @@ int machine__process_fork_event(struct machine *machine, union perf_event *event
         return 0;
 }
 
+static void machine__remove_thread(struct machine *machine, struct thread *th)
+{
+        machine->last_match = NULL;
+        rb_erase(&th->rb_node, &machine->threads);
+        /*
+         * We may have references to this thread, for instance in some hist_entry
+         * instances, so just move them to a separate list.
+         */
+        list_add_tail(&th->node, &machine->dead_threads);
+}
+
 int machine__process_exit_event(struct machine *machine, union perf_event *event)
 {
         struct thread *thread = machine__find_thread(machine, event->fork.tid);
@@ -1039,17 +1058,6 @@ int machine__process_event(struct machine *machine, union perf_event *event)
         return ret;
 }
 
-void machine__remove_thread(struct machine *machine, struct thread *th)
-{
-        machine->last_match = NULL;
-        rb_erase(&th->rb_node, &machine->threads);
-        /*
-         * We may have references to this thread, for instance in some hist_entry
-         * instances, so just move them to a separate list.
-         */
-        list_add_tail(&th->node, &machine->dead_threads);
-}
-
 static bool symbol__match_parent_regex(struct symbol *sym)
 {
         if (sym->name && !regexec(&parent_regex, sym->name, 0, NULL, 0))
@@ -1097,6 +1105,38 @@ found:
         ams->map = al.map;
 }
 
+static void ip__resolve_data(struct machine *machine, struct thread *thread,
+                             u8 m, struct addr_map_symbol *ams, u64 addr)
+{
+        struct addr_location al;
+
+        memset(&al, 0, sizeof(al));
+
+        thread__find_addr_location(thread, machine, m, MAP__VARIABLE, addr, &al,
+                                   NULL);
+        ams->addr = addr;
+        ams->al_addr = al.addr;
+        ams->sym = al.sym;
+        ams->map = al.map;
+}
+
+struct mem_info *machine__resolve_mem(struct machine *machine,
+                                      struct thread *thr,
+                                      struct perf_sample *sample,
+                                      u8 cpumode)
+{
+        struct mem_info *mi = zalloc(sizeof(*mi));
+
+        if (!mi)
+                return NULL;
+
+        ip__resolve_ams(machine, thr, &mi->iaddr, sample->ip);
+        ip__resolve_data(machine, thr, cpumode, &mi->daddr, sample->addr);
+        mi->data_src.val = sample->data_src;
+
+        return mi;
+}
+
 struct branch_info *machine__resolve_bstack(struct machine *machine,
                                             struct thread *thr,
                                             struct branch_stack *bs)
diff --git a/tools/perf/util/machine.h b/tools/perf/util/machine.h
index 5ac5892f2326..77940680f1fc 100644
--- a/tools/perf/util/machine.h
+++ b/tools/perf/util/machine.h
@@ -76,6 +76,9 @@ void machine__delete(struct machine *machine);
 struct branch_info *machine__resolve_bstack(struct machine *machine,
                                             struct thread *thread,
                                             struct branch_stack *bs);
+struct mem_info *machine__resolve_mem(struct machine *machine,
+                                      struct thread *thread,
+                                      struct perf_sample *sample, u8 cpumode);
 int machine__resolve_callchain(struct machine *machine,
                                struct perf_evsel *evsel,
                                struct thread *thread,
@@ -97,7 +100,6 @@ static inline bool machine__is_host(struct machine *machine)
 }
 
 struct thread *machine__findnew_thread(struct machine *machine, pid_t pid);
-void machine__remove_thread(struct machine *machine, struct thread *th);
 
 size_t machine__fprintf(struct machine *machine, FILE *fp);
 
diff --git a/tools/perf/util/parse-events.c b/tools/perf/util/parse-events.c
index c84f48cf9678..6c8bb0fb189b 100644
--- a/tools/perf/util/parse-events.c
+++ b/tools/perf/util/parse-events.c
@@ -10,7 +10,7 @@
 #include "symbol.h"
 #include "cache.h"
 #include "header.h"
-#include "debugfs.h"
+#include <lk/debugfs.h>
 #include "parse-events-bison.h"
 #define YY_EXTRA_TYPE int
 #include "parse-events-flex.h"
diff --git a/tools/perf/util/probe-event.c b/tools/perf/util/probe-event.c
index 49a256e6e0a2..aa04bf9c9ad7 100644
--- a/tools/perf/util/probe-event.c
+++ b/tools/perf/util/probe-event.c
@@ -40,7 +40,7 @@
 #include "color.h"
 #include "symbol.h"
 #include "thread.h"
-#include "debugfs.h"
+#include <lk/debugfs.h>
 #include "trace-event.h"        /* For __maybe_unused */
 #include "probe-event.h"
 #include "probe-finder.h"
diff --git a/tools/perf/util/python-ext-sources b/tools/perf/util/python-ext-sources
index 64536a993f4a..f75ae1b9900c 100644
--- a/tools/perf/util/python-ext-sources
+++ b/tools/perf/util/python-ext-sources
@@ -15,7 +15,6 @@ util/thread_map.c
 util/util.c
 util/xyarray.c
 util/cgroup.c
-util/debugfs.c
 util/rblist.c
 util/strlist.c
 util/sysfs.c
diff --git a/tools/perf/util/session.c b/tools/perf/util/session.c
index bd85280bb6e8..cf1fe01b7e89 100644
--- a/tools/perf/util/session.c
+++ b/tools/perf/util/session.c
@@ -1,5 +1,3 @@
-#define _FILE_OFFSET_BITS 64
-
 #include <linux/kernel.h>
 
 #include <byteswap.h>
@@ -800,6 +798,12 @@ static void dump_sample(struct perf_evsel *evsel, union perf_event *event,
 
         if (sample_type & PERF_SAMPLE_STACK_USER)
                 stack_user__printf(&sample->user_stack);
+
+        if (sample_type & PERF_SAMPLE_WEIGHT)
+                printf("... weight: %" PRIu64 "\n", sample->weight);
+
+        if (sample_type & PERF_SAMPLE_DATA_SRC)
+                printf(" . data_src: 0x%"PRIx64"\n", sample->data_src);
 }
 
 static struct machine *
@@ -1365,18 +1369,6 @@ size_t perf_session__fprintf(struct perf_session *session, FILE *fp)
         return machine__fprintf(&session->machines.host, fp);
 }
 
-void perf_session__remove_thread(struct perf_session *session,
-                                 struct thread *th)
-{
-        /*
-         * FIXME: This one makes no sense, we need to remove the thread from
-         * the machine it belongs to, perf_session can have many machines, so
-         * doing it always on ->machines.host is wrong.  Fix when auditing all
-         * the 'perf kvm' code.
-         */
-        machine__remove_thread(&session->machines.host, th);
-}
-
 struct perf_evsel *perf_session__find_first_evtype(struct perf_session *session,
                                               unsigned int type)
 {
diff --git a/tools/perf/util/session.h b/tools/perf/util/session.h
index b5c0847edfa9..6b51d47acdba 100644
--- a/tools/perf/util/session.h
+++ b/tools/perf/util/session.h
@@ -72,7 +72,6 @@ void perf_event__attr_swap(struct perf_event_attr *attr);
 int perf_session__create_kernel_maps(struct perf_session *self);
 
 void perf_session__set_id_hdr_size(struct perf_session *session);
-void perf_session__remove_thread(struct perf_session *self, struct thread *th);
 
 static inline
 struct machine *perf_session__find_machine(struct perf_session *self, pid_t pid)
diff --git a/tools/perf/util/setup.py b/tools/perf/util/setup.py
index 73d510269784..6b0ed322907e 100644
--- a/tools/perf/util/setup.py
+++ b/tools/perf/util/setup.py
@@ -24,6 +24,7 @@ cflags += getenv('CFLAGS', '').split()
 build_lib = getenv('PYTHON_EXTBUILD_LIB')
 build_tmp = getenv('PYTHON_EXTBUILD_TMP')
 libtraceevent = getenv('LIBTRACEEVENT')
+liblk = getenv('LIBLK')
 
 ext_sources = [f.strip() for f in file('util/python-ext-sources')
                                 if len(f.strip()) > 0 and f[0] != '#']
@@ -32,7 +33,7 @@ perf = Extension('perf',
                   sources = ext_sources,
                   include_dirs = ['util/include'],
                   extra_compile_args = cflags,
-                  extra_objects = [libtraceevent],
+                  extra_objects = [libtraceevent, liblk],
                  )
 
 setup(name='perf',
diff --git a/tools/perf/util/sort.c b/tools/perf/util/sort.c
index d41926cb9e3f..5f52d492590c 100644
--- a/tools/perf/util/sort.c
+++ b/tools/perf/util/sort.c
@@ -198,11 +198,19 @@ static int _hist_entry__sym_snprintf(struct map *map, struct symbol *sym,
         }
 
         ret += repsep_snprintf(bf + ret, size - ret, "[%c] ", level);
-        if (sym)
-                ret += repsep_snprintf(bf + ret, size - ret, "%-*s",
-                                       width - ret,
-                                       sym->name);
-        else {
+        if (sym && map) {
+                if (map->type == MAP__VARIABLE) {
+                        ret += repsep_snprintf(bf + ret, size - ret, "%s", sym->name);
+                        ret += repsep_snprintf(bf + ret, size - ret, "+0x%llx",
+                                        ip - map->unmap_ip(map, sym->start));
+                        ret += repsep_snprintf(bf + ret, size - ret, "%-*s",
+                                       width - ret, "");
+                } else {
+                        ret += repsep_snprintf(bf + ret, size - ret, "%-*s",
+                                               width - ret,
+                                               sym->name);
+                }
+        } else {
                 size_t len = BITS_PER_LONG / 4;
                 ret += repsep_snprintf(bf + ret, size - ret, "%-#.*llx",
                                        len, ip);
@@ -457,6 +465,304 @@ static int hist_entry__mispredict_snprintf(struct hist_entry *self, char *bf,
         return repsep_snprintf(bf, size, "%-*s", width, out);
 }
 
+/* --sort daddr_sym */
+static int64_t
+sort__daddr_cmp(struct hist_entry *left, struct hist_entry *right)
+{
+        uint64_t l = 0, r = 0;
+
+        if (left->mem_info)
+                l = left->mem_info->daddr.addr;
+        if (right->mem_info)
+                r = right->mem_info->daddr.addr;
+
+        return (int64_t)(r - l);
+}
+
+static int hist_entry__daddr_snprintf(struct hist_entry *self, char *bf,
+                                    size_t size, unsigned int width)
+{
+        uint64_t addr = 0;
+        struct map *map = NULL;
+        struct symbol *sym = NULL;
+
+        if (self->mem_info) {
+                addr = self->mem_info->daddr.addr;
+                map = self->mem_info->daddr.map;
+                sym = self->mem_info->daddr.sym;
+        }
+        return _hist_entry__sym_snprintf(map, sym, addr, self->level, bf, size,
+                                         width);
+}
+
+static int64_t
+sort__dso_daddr_cmp(struct hist_entry *left, struct hist_entry *right)
+{
+        struct map *map_l = NULL;
+        struct map *map_r = NULL;
+
+        if (left->mem_info)
+                map_l = left->mem_info->daddr.map;
+        if (right->mem_info)
+                map_r = right->mem_info->daddr.map;
+
+        return _sort__dso_cmp(map_l, map_r);
+}
+
+static int hist_entry__dso_daddr_snprintf(struct hist_entry *self, char *bf,
+                                    size_t size, unsigned int width)
+{
+        struct map *map = NULL;
+
+        if (self->mem_info)
+                map = self->mem_info->daddr.map;
+
+        return _hist_entry__dso_snprintf(map, bf, size, width);
+}
+
+static int64_t
+sort__locked_cmp(struct hist_entry *left, struct hist_entry *right)
+{
+        union perf_mem_data_src data_src_l;
+        union perf_mem_data_src data_src_r;
+
+        if (left->mem_info)
+                data_src_l = left->mem_info->data_src;
+        else
+                data_src_l.mem_lock = PERF_MEM_LOCK_NA;
+
+        if (right->mem_info)
+                data_src_r = right->mem_info->data_src;
+        else
+                data_src_r.mem_lock = PERF_MEM_LOCK_NA;
+
+        return (int64_t)(data_src_r.mem_lock - data_src_l.mem_lock);
+}
+
+static int hist_entry__locked_snprintf(struct hist_entry *self, char *bf,
+                                    size_t size, unsigned int width)
+{
+        const char *out;
+        u64 mask = PERF_MEM_LOCK_NA;
+
+        if (self->mem_info)
+                mask = self->mem_info->data_src.mem_lock;
+
+        if (mask & PERF_MEM_LOCK_NA)
+                out = "N/A";
+        else if (mask & PERF_MEM_LOCK_LOCKED)
+                out = "Yes";
+        else
+                out = "No";
+
+        return repsep_snprintf(bf, size, "%-*s", width, out);
+}
+
+static int64_t
+sort__tlb_cmp(struct hist_entry *left, struct hist_entry *right)
+{
+        union perf_mem_data_src data_src_l;
+        union perf_mem_data_src data_src_r;
+
+        if (left->mem_info)
+                data_src_l = left->mem_info->data_src;
+        else
+                data_src_l.mem_dtlb = PERF_MEM_TLB_NA;
+
+        if (right->mem_info)
+                data_src_r = right->mem_info->data_src;
+        else
+                data_src_r.mem_dtlb = PERF_MEM_TLB_NA;
+
+        return (int64_t)(data_src_r.mem_dtlb - data_src_l.mem_dtlb);
+}
+
+static const char * const tlb_access[] = {
+        "N/A",
+        "HIT",
+        "MISS",
+        "L1",
+        "L2",
+        "Walker",
+        "Fault",
+};
+#define NUM_TLB_ACCESS (sizeof(tlb_access)/sizeof(const char *))
+
+static int hist_entry__tlb_snprintf(struct hist_entry *self, char *bf,
+                                    size_t size, unsigned int width)
+{
+        char out[64];
+        size_t sz = sizeof(out) - 1; /* -1 for null termination */
+        size_t l = 0, i;
+        u64 m = PERF_MEM_TLB_NA;
+        u64 hit, miss;
+
+        out[0] = '\0';
+
+        if (self->mem_info)
+                m = self->mem_info->data_src.mem_dtlb;
+
+        hit = m & PERF_MEM_TLB_HIT;
+        miss = m & PERF_MEM_TLB_MISS;
+
+        /* already taken care of */
+        m &= ~(PERF_MEM_TLB_HIT|PERF_MEM_TLB_MISS);
+
+        for (i = 0; m && i < NUM_TLB_ACCESS; i++, m >>= 1) {
+                if (!(m & 0x1))
+                        continue;
+                if (l) {
+                        strcat(out, " or ");
+                        l += 4;
+                }
+                strncat(out, tlb_access[i], sz - l);
+                l += strlen(tlb_access[i]);
+        }
+        if (*out == '\0')
+                strcpy(out, "N/A");
+        if (hit)
+                strncat(out, " hit", sz - l);
+        if (miss)
+                strncat(out, " miss", sz - l);
+
+        return repsep_snprintf(bf, size, "%-*s", width, out);
+}
+
+static int64_t
+sort__lvl_cmp(struct hist_entry *left, struct hist_entry *right)
+{
+        union perf_mem_data_src data_src_l;
+        union perf_mem_data_src data_src_r;
+
+        if (left->mem_info)
+                data_src_l = left->mem_info->data_src;
+        else
+                data_src_l.mem_lvl = PERF_MEM_LVL_NA;
+
+        if (right->mem_info)
+                data_src_r = right->mem_info->data_src;
+        else
+                data_src_r.mem_lvl = PERF_MEM_LVL_NA;
+
+        return (int64_t)(data_src_r.mem_lvl - data_src_l.mem_lvl);
+}
+
+static const char * const mem_lvl[] = {
+        "N/A",
+        "HIT",
+        "MISS",
+        "L1",
+        "LFB",
+        "L2",
+        "L3",
+        "Local RAM",
+        "Remote RAM (1 hop)",
+        "Remote RAM (2 hops)",
+        "Remote Cache (1 hop)",
+        "Remote Cache (2 hops)",
+        "I/O",
+        "Uncached",
+};
+#define NUM_MEM_LVL (sizeof(mem_lvl)/sizeof(const char *))
+
+static int hist_entry__lvl_snprintf(struct hist_entry *self, char *bf,
+                                    size_t size, unsigned int width)
+{
+        char out[64];
+        size_t sz = sizeof(out) - 1; /* -1 for null termination */
+        size_t i, l = 0;
+        u64 m =  PERF_MEM_LVL_NA;
+        u64 hit, miss;
+
+        if (self->mem_info)
+                m  = self->mem_info->data_src.mem_lvl;
+
+        out[0] = '\0';
+
+        hit = m & PERF_MEM_LVL_HIT;
+        miss = m & PERF_MEM_LVL_MISS;
+
+        /* already taken care of */
+        m &= ~(PERF_MEM_LVL_HIT|PERF_MEM_LVL_MISS);
+
+        for (i = 0; m && i < NUM_MEM_LVL; i++, m >>= 1) {
+                if (!(m & 0x1))
+                        continue;
+                if (l) {
+                        strcat(out, " or ");
+                        l += 4;
+                }
+                strncat(out, mem_lvl[i], sz - l);
+                l += strlen(mem_lvl[i]);
+        }
+        if (*out == '\0')
+                strcpy(out, "N/A");
+        if (hit)
+                strncat(out, " hit", sz - l);
+        if (miss)
+                strncat(out, " miss", sz - l);
+
+        return repsep_snprintf(bf, size, "%-*s", width, out);
+}
+
+static int64_t
+sort__snoop_cmp(struct hist_entry *left, struct hist_entry *right)
+{
+        union perf_mem_data_src data_src_l;
+        union perf_mem_data_src data_src_r;
+
+        if (left->mem_info)
+                data_src_l = left->mem_info->data_src;
+        else
+                data_src_l.mem_snoop = PERF_MEM_SNOOP_NA;
+
+        if (right->mem_info)
+                data_src_r = right->mem_info->data_src;
+        else
+                data_src_r.mem_snoop = PERF_MEM_SNOOP_NA;
+
+        return (int64_t)(data_src_r.mem_snoop - data_src_l.mem_snoop);
+}
+
+static const char * const snoop_access[] = {
+        "N/A",
+        "None",
+        "Miss",
+        "Hit",
+        "HitM",
+};
+#define NUM_SNOOP_ACCESS (sizeof(snoop_access)/sizeof(const char *))
+
+static int hist_entry__snoop_snprintf(struct hist_entry *self, char *bf,
+                                    size_t size, unsigned int width)
+{
+        char out[64];
+        size_t sz = sizeof(out) - 1; /* -1 for null termination */
+        size_t i, l = 0;
+        u64 m = PERF_MEM_SNOOP_NA;
+
+        out[0] = '\0';
+
+        if (self->mem_info)
+                m = self->mem_info->data_src.mem_snoop;
+
+        for (i = 0; m && i < NUM_SNOOP_ACCESS; i++, m >>= 1) {
+                if (!(m & 0x1))
+                        continue;
+                if (l) {
+                        strcat(out, " or ");
+                        l += 4;
+                }
+                strncat(out, snoop_access[i], sz - l);
+                l += strlen(snoop_access[i]);
+        }
+
+        if (*out == '\0')
+                strcpy(out, "N/A");
+
+        return repsep_snprintf(bf, size, "%-*s", width, out);
+}
+
 struct sort_entry sort_mispredict = {
         .se_header      = "Branch Mispredicted",
         .se_cmp         = sort__mispredict_cmp,
@@ -464,6 +770,91 @@ struct sort_entry sort_mispredict = {
         .se_width_idx   = HISTC_MISPREDICT,
 };
 
+static u64 he_weight(struct hist_entry *he)
+{
+        return he->stat.nr_events ? he->stat.weight / he->stat.nr_events : 0;
+}
+
+static int64_t
+sort__local_weight_cmp(struct hist_entry *left, struct hist_entry *right)
+{
+        return he_weight(left) - he_weight(right);
+}
+
+static int hist_entry__local_weight_snprintf(struct hist_entry *self, char *bf,
+                                    size_t size, unsigned int width)
+{
+        return repsep_snprintf(bf, size, "%-*llu", width, he_weight(self));
+}
+
+struct sort_entry sort_local_weight = {
+        .se_header      = "Local Weight",
+        .se_cmp         = sort__local_weight_cmp,
+        .se_snprintf    = hist_entry__local_weight_snprintf,
+        .se_width_idx   = HISTC_LOCAL_WEIGHT,
+};
+
+static int64_t
+sort__global_weight_cmp(struct hist_entry *left, struct hist_entry *right)
+{
+        return left->stat.weight - right->stat.weight;
+}
+
+static int hist_entry__global_weight_snprintf(struct hist_entry *self, char *bf,
+                                              size_t size, unsigned int width)
+{
+        return repsep_snprintf(bf, size, "%-*llu", width, self->stat.weight);
+}
+
+struct sort_entry sort_global_weight = {
+        .se_header      = "Weight",
+        .se_cmp         = sort__global_weight_cmp,
+        .se_snprintf    = hist_entry__global_weight_snprintf,
+        .se_width_idx   = HISTC_GLOBAL_WEIGHT,
+};
+
+struct sort_entry sort_mem_daddr_sym = {
+        .se_header      = "Data Symbol",
+        .se_cmp         = sort__daddr_cmp,
+        .se_snprintf    = hist_entry__daddr_snprintf,
+        .se_width_idx   = HISTC_MEM_DADDR_SYMBOL,
+};
+
+struct sort_entry sort_mem_daddr_dso = {
+        .se_header      = "Data Object",
+        .se_cmp         = sort__dso_daddr_cmp,
+        .se_snprintf    = hist_entry__dso_daddr_snprintf,
+        .se_width_idx   = HISTC_MEM_DADDR_SYMBOL,
+};
+
+struct sort_entry sort_mem_locked = {
+        .se_header      = "Locked",
+        .se_cmp         = sort__locked_cmp,
+        .se_snprintf    = hist_entry__locked_snprintf,
+        .se_width_idx   = HISTC_MEM_LOCKED,
+};
+
+struct sort_entry sort_mem_tlb = {
+        .se_header      = "TLB access",
+        .se_cmp         = sort__tlb_cmp,
+        .se_snprintf    = hist_entry__tlb_snprintf,
+        .se_width_idx   = HISTC_MEM_TLB,
+};
+
+struct sort_entry sort_mem_lvl = {
+        .se_header      = "Memory access",
+        .se_cmp         = sort__lvl_cmp,
+        .se_snprintf    = hist_entry__lvl_snprintf,
+        .se_width_idx   = HISTC_MEM_LVL,
+};
+
+struct sort_entry sort_mem_snoop = {
+        .se_header      = "Snoop",
+        .se_cmp         = sort__snoop_cmp,
+        .se_snprintf    = hist_entry__snoop_snprintf,
+        .se_width_idx   = HISTC_MEM_SNOOP,
+};
+
 struct sort_dimension {
         const char              *name;
         struct sort_entry       *entry;
@@ -480,6 +871,14 @@ static struct sort_dimension common_sort_dimensions[] = {
         DIM(SORT_PARENT, "parent", sort_parent),
         DIM(SORT_CPU, "cpu", sort_cpu),
         DIM(SORT_SRCLINE, "srcline", sort_srcline),
+        DIM(SORT_LOCAL_WEIGHT, "local_weight", sort_local_weight),
+        DIM(SORT_GLOBAL_WEIGHT, "weight", sort_global_weight),
+        DIM(SORT_MEM_DADDR_SYMBOL, "symbol_daddr", sort_mem_daddr_sym),
+        DIM(SORT_MEM_DADDR_DSO, "dso_daddr", sort_mem_daddr_dso),
+        DIM(SORT_MEM_LOCKED, "locked", sort_mem_locked),
+        DIM(SORT_MEM_TLB, "tlb", sort_mem_tlb),
+        DIM(SORT_MEM_LVL, "mem", sort_mem_lvl),
+        DIM(SORT_MEM_SNOOP, "snoop", sort_mem_snoop),
 };
 
 #undef DIM
@@ -516,7 +915,10 @@ int sort_dimension__add(const char *tok)
                                 return -EINVAL;
                         }
                         sort__has_parent = 1;
-                } else if (sd->entry == &sort_sym) {
+                } else if (sd->entry == &sort_sym ||
+                           sd->entry == &sort_sym_from ||
+                           sd->entry == &sort_sym_to ||
+                           sd->entry == &sort_mem_daddr_sym) {
                         sort__has_sym = 1;
                 }
 
diff --git a/tools/perf/util/sort.h b/tools/perf/util/sort.h
index b13e56f6ccbe..f24bdf64238c 100644
--- a/tools/perf/util/sort.h
+++ b/tools/perf/util/sort.h
@@ -49,6 +49,7 @@ struct he_stat {
         u64                     period_us;
         u64                     period_guest_sys;
         u64                     period_guest_us;
+        u64                     weight;
         u32                     nr_events;
 };
 
@@ -100,7 +101,8 @@ struct hist_entry {
         struct rb_root          sorted_chain;
         struct branch_info      *branch_info;
         struct hists            *hists;
-        struct callchain_root   callchain[0];
+        struct mem_info         *mem_info;
+        struct callchain_root   callchain[0]; /* must be last member */
 };
 
 static inline bool hist_entry__has_pairs(struct hist_entry *he)
@@ -130,6 +132,14 @@ enum sort_type {
         SORT_PARENT,
         SORT_CPU,
         SORT_SRCLINE,
+        SORT_LOCAL_WEIGHT,
+        SORT_GLOBAL_WEIGHT,
+        SORT_MEM_DADDR_SYMBOL,
+        SORT_MEM_DADDR_DSO,
+        SORT_MEM_LOCKED,
+        SORT_MEM_TLB,
+        SORT_MEM_LVL,
+        SORT_MEM_SNOOP,
 
         /* branch stack specific sort keys */
         __SORT_BRANCH_STACK,
diff --git a/tools/perf/util/symbol-elf.c b/tools/perf/util/symbol-elf.c
index 54efcb5659ac..4b12bf850325 100644
--- a/tools/perf/util/symbol-elf.c
+++ b/tools/perf/util/symbol-elf.c
@@ -806,9 +806,12 @@ int dso__load_sym(struct dso *dso, struct map *map,
                  * DWARF DW_compile_unit has this, but we don't always have access
                  * to it...
                  */
-                demangled = bfd_demangle(NULL, elf_name, DMGL_PARAMS | DMGL_ANSI);
-                if (demangled != NULL)
-                        elf_name = demangled;
+                if (symbol_conf.demangle) {
+                        demangled = bfd_demangle(NULL, elf_name,
+                                                 DMGL_PARAMS | DMGL_ANSI);
+                        if (demangled != NULL)
+                                elf_name = demangled;
+                }
 new_symbol:
                 f = symbol__new(sym.st_value, sym.st_size,
                                 GELF_ST_BIND(sym.st_info), elf_name);
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index e6432d85b43d..8cf3b5426a9a 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -36,6 +36,7 @@ struct symbol_conf symbol_conf = {
         .use_modules      = true,
         .try_vmlinux_path = true,
         .annotate_src     = true,
+        .demangle         = true,
         .symfs            = "",
 };
 
diff --git a/tools/perf/util/symbol.h b/tools/perf/util/symbol.h
index b62ca37c4b77..5f720dc076da 100644
--- a/tools/perf/util/symbol.h
+++ b/tools/perf/util/symbol.h
@@ -97,7 +97,8 @@ struct symbol_conf {
                         kptr_restrict,
                         annotate_asm_raw,
                         annotate_src,
-                        event_group;
+                        event_group,
+                        demangle;
         const char      *vmlinux_name,
                         *kallsyms_name,
                         *source_prefix,
@@ -155,6 +156,12 @@ struct branch_info {
         struct branch_flags flags;
 };
 
+struct mem_info {
+        struct addr_map_symbol iaddr;
+        struct addr_map_symbol daddr;
+        union perf_mem_data_src data_src;
+};
+
 struct addr_location {
         struct thread *thread;
         struct map    *map;
diff --git a/tools/perf/util/thread_map.h b/tools/perf/util/thread_map.h
index f718df8a3c59..0cd8b3108084 100644
--- a/tools/perf/util/thread_map.h
+++ b/tools/perf/util/thread_map.h
@@ -21,4 +21,9 @@ void thread_map__delete(struct thread_map *threads);
 
 size_t thread_map__fprintf(struct thread_map *threads, FILE *fp);
 
+static inline int thread_map__nr(struct thread_map *threads)
+{
+        return threads ? threads->nr : 1;
+}
+
 #endif  /* __PERF_THREAD_MAP_H */
diff --git a/tools/perf/util/trace-event-info.c b/tools/perf/util/trace-event-info.c
index a8d81c35ef66..3917eb9a8479 100644
--- a/tools/perf/util/trace-event-info.c
+++ b/tools/perf/util/trace-event-info.c
@@ -38,52 +38,20 @@
 
 #include "../perf.h"
 #include "trace-event.h"
-#include "debugfs.h"
+#include <lk/debugfs.h>
 #include "evsel.h"
 
 #define VERSION "0.5"
 
-#define TRACE_CTRL      "tracing_on"
-#define TRACE           "trace"
-#define AVAILABLE       "available_tracers"
-#define CURRENT         "current_tracer"
-#define ITER_CTRL       "trace_options"
-#define MAX_LATENCY     "tracing_max_latency"
-
-unsigned int page_size;
-
-static const char *output_file = "trace.info";
 static int output_fd;
 
-struct event_list {
-        struct event_list *next;
-        const char *event;
-};
-
-struct events {
-        struct events *sibling;
-        struct events *children;
-        struct events *next;
-        char *name;
-};
-
-
-static void *malloc_or_die(unsigned int size)
-{
-        void *data;
-
-        data = malloc(size);
-        if (!data)
-                die("malloc");
-        return data;
-}
 
 static const char *find_debugfs(void)
 {
-        const char *path = debugfs_mount(NULL);
+        const char *path = perf_debugfs_mount(NULL);
 
         if (!path)
-                die("Your kernel not support debugfs filesystem");
+                pr_debug("Your kernel does not support the debugfs filesystem");
 
         return path;
 }
@@ -102,8 +70,12 @@ static const char *find_tracing_dir(void)
                 return tracing;
 
         debugfs = find_debugfs();
+        if (!debugfs)
+                return NULL;
 
-        tracing = malloc_or_die(strlen(debugfs) + 9);
+        tracing = malloc(strlen(debugfs) + 9);
+        if (!tracing)
+                return NULL;
 
         sprintf(tracing, "%s/tracing", debugfs);
 
@@ -120,7 +92,9 @@ static char *get_tracing_file(const char *name)
         if (!tracing)
                 return NULL;
 
-        file = malloc_or_die(strlen(tracing) + strlen(name) + 2);
+        file = malloc(strlen(tracing) + strlen(name) + 2);
+        if (!file)
+                return NULL;
 
         sprintf(file, "%s/%s", tracing, name);
         return file;
@@ -131,24 +105,6 @@ static void put_tracing_file(char *file)
         free(file);
 }
 
-static ssize_t calc_data_size;
-
-static ssize_t write_or_die(const void *buf, size_t len)
-{
-        int ret;
-
-        if (calc_data_size) {
-                calc_data_size += len;
-                return len;
-        }
-
-        ret = write(output_fd, buf, len);
-        if (ret < 0)
-                die("writing to '%s'", output_file);
-
-        return ret;
-}
-
 int bigendian(void)
 {
         unsigned char str[] = { 0x1, 0x2, 0x3, 0x4, 0x0, 0x0, 0x0, 0x0};
@@ -159,59 +115,106 @@ int bigendian(void)
 }
 
 /* unfortunately, you can not stat debugfs or proc files for size */
-static void record_file(const char *file, size_t hdr_sz)
+static int record_file(const char *file, ssize_t hdr_sz)
 {
         unsigned long long size = 0;
         char buf[BUFSIZ], *sizep;
         off_t hdr_pos = lseek(output_fd, 0, SEEK_CUR);
         int r, fd;
+        int err = -EIO;
 
         fd = open(file, O_RDONLY);
-        if (fd < 0)
-                die("Can't read '%s'", file);
+        if (fd < 0) {
+                pr_debug("Can't read '%s'", file);
+                return -errno;
+        }
 
         /* put in zeros for file size, then fill true size later */
-        if (hdr_sz)
-                write_or_die(&size, hdr_sz);
+        if (hdr_sz) {
+                if (write(output_fd, &size, hdr_sz) != hdr_sz)
+                        goto out;
+        }
 
         do {
                 r = read(fd, buf, BUFSIZ);
                 if (r > 0) {
                         size += r;
-                        write_or_die(buf, r);
+                        if (write(output_fd, buf, r) != r)
+                                goto out;
                 }
         } while (r > 0);
-        close(fd);
 
         /* ugh, handle big-endian hdr_size == 4 */
         sizep = (char*)&size;
         if (bigendian())
                 sizep += sizeof(u64) - hdr_sz;
 
-        if (hdr_sz && pwrite(output_fd, sizep, hdr_sz, hdr_pos) < 0)
-                die("writing to %s", output_file);
+        if (hdr_sz && pwrite(output_fd, sizep, hdr_sz, hdr_pos) < 0) {
+                pr_debug("writing file size failed\n");
+                goto out;
+        }
+
+        err = 0;
+out:
+        close(fd);
+        return err;
 }
 
-static void read_header_files(void)
+static int read_header_files(void)
 {
         char *path;
         struct stat st;
+        int err = -EIO;
 
         path = get_tracing_file("events/header_page");
-        if (stat(path, &st) < 0)
-                die("can't read '%s'", path);
+        if (!path) {
+                pr_debug("can't get tracing/events/header_page");
+                return -ENOMEM;
+        }
+
+        if (stat(path, &st) < 0) {
+                pr_debug("can't read '%s'", path);
+                goto out;
+        }
+
+        if (write(output_fd, "header_page", 12) != 12) {
+                pr_debug("can't write header_page\n");
+                goto out;
+        }
+
+        if (record_file(path, 8) < 0) {
+                pr_debug("can't record header_page file\n");
+                goto out;
+        }
 
-        write_or_die("header_page", 12);
-        record_file(path, 8);
         put_tracing_file(path);
 
         path = get_tracing_file("events/header_event");
-        if (stat(path, &st) < 0)
-                die("can't read '%s'", path);
+        if (!path) {
+                pr_debug("can't get tracing/events/header_event");
+                err = -ENOMEM;
+                goto out;
+        }
+
+        if (stat(path, &st) < 0) {
+                pr_debug("can't read '%s'", path);
+                goto out;
+        }
 
-        write_or_die("header_event", 13);
-        record_file(path, 8);
+        if (write(output_fd, "header_event", 13) != 13) {
+                pr_debug("can't write header_event\n");
+                goto out;
+        }
+
+        if (record_file(path, 8) < 0) {
+                pr_debug("can't record header_event file\n");
+                goto out;
+        }
+
+        err = 0;
+out:
         put_tracing_file(path);
+        return err;
 }
 
 static bool name_in_tp_list(char *sys, struct tracepoint_path *tps)
@@ -225,7 +228,7 @@ static bool name_in_tp_list(char *sys, struct tracepoint_path *tps)
         return false;
 }
 
-static void copy_event_system(const char *sys, struct tracepoint_path *tps)
+static int copy_event_system(const char *sys, struct tracepoint_path *tps)
 {
         struct dirent *dent;
         struct stat st;
@@ -233,10 +236,13 @@ static void copy_event_system(const char *sys, struct tracepoint_path *tps)
         DIR *dir;
         int count = 0;
         int ret;
+        int err;
 
         dir = opendir(sys);
-        if (!dir)
-                die("can't read directory '%s'", sys);
+        if (!dir) {
+                pr_debug("can't read directory '%s'", sys);
+                return -errno;
+        }
 
         while ((dent = readdir(dir))) {
                 if (dent->d_type != DT_DIR ||
@@ -244,7 +250,11 @@ static void copy_event_system(const char *sys, struct tracepoint_path *tps)
                     strcmp(dent->d_name, "..") == 0 ||
                     !name_in_tp_list(dent->d_name, tps))
                         continue;
-                format = malloc_or_die(strlen(sys) + strlen(dent->d_name) + 10);
+                format = malloc(strlen(sys) + strlen(dent->d_name) + 10);
+                if (!format) {
+                        err = -ENOMEM;
+                        goto out;
+                }
                 sprintf(format, "%s/%s/format", sys, dent->d_name);
                 ret = stat(format, &st);
                 free(format);
@@ -253,7 +263,11 @@ static void copy_event_system(const char *sys, struct tracepoint_path *tps)
                 count++;
         }
 
-        write_or_die(&count, 4);
+        if (write(output_fd, &count, 4) != 4) {
+                err = -EIO;
+                pr_debug("can't write count\n");
+                goto out;
+        }
 
         rewinddir(dir);
         while ((dent = readdir(dir))) {
@@ -262,27 +276,45 @@ static void copy_event_system(const char *sys, struct tracepoint_path *tps)
                     strcmp(dent->d_name, "..") == 0 ||
                     !name_in_tp_list(dent->d_name, tps))
                         continue;
-                format = malloc_or_die(strlen(sys) + strlen(dent->d_name) + 10);
+                format = malloc(strlen(sys) + strlen(dent->d_name) + 10);
+                if (!format) {
+                        err = -ENOMEM;
+                        goto out;
+                }
                 sprintf(format, "%s/%s/format", sys, dent->d_name);
                 ret = stat(format, &st);
 
-                if (ret >= 0)
-                        record_file(format, 8);
-
+                if (ret >= 0) {
+                        err = record_file(format, 8);
+                        if (err) {
+                                free(format);
+                                goto out;
+                        }
+                }
                 free(format);
         }
+        err = 0;
+out:
         closedir(dir);
+        return err;
 }
 
-static void read_ftrace_files(struct tracepoint_path *tps)
+static int read_ftrace_files(struct tracepoint_path *tps)
 {
         char *path;
+        int ret;
 
         path = get_tracing_file("events/ftrace");
+        if (!path) {
+                pr_debug("can't get tracing/events/ftrace");
+                return -ENOMEM;
+        }
 
-        copy_event_system(path, tps);
+        ret = copy_event_system(path, tps);
 
         put_tracing_file(path);
+
+        return ret;
 }
 
 static bool system_in_tp_list(char *sys, struct tracepoint_path *tps)
@@ -296,7 +328,7 @@ static bool system_in_tp_list(char *sys, struct tracepoint_path *tps)
         return false;
 }
 
-static void read_event_files(struct tracepoint_path *tps)
+static int read_event_files(struct tracepoint_path *tps)
 {
         struct dirent *dent;
         struct stat st;
@@ -305,12 +337,20 @@ static void read_event_files(struct tracepoint_path *tps)
         DIR *dir;
         int count = 0;
         int ret;
+        int err;
 
         path = get_tracing_file("events");
+        if (!path) {
+                pr_debug("can't get tracing/events");
+                return -ENOMEM;
+        }
 
         dir = opendir(path);
-        if (!dir)
-                die("can't read directory '%s'", path);
+        if (!dir) {
+                err = -errno;
+                pr_debug("can't read directory '%s'", path);
+                goto out;
+        }
 
         while ((dent = readdir(dir))) {
                 if (dent->d_type != DT_DIR ||
@@ -322,7 +362,11 @@ static void read_event_files(struct tracepoint_path *tps)
                 count++;
         }
 
-        write_or_die(&count, 4);
+        if (write(output_fd, &count, 4) != 4) {
+                err = -EIO;
+                pr_debug("can't write count\n");
+                goto out;
+        }
 
         rewinddir(dir);
         while ((dent = readdir(dir))) {
@@ -332,56 +376,90 @@ static void read_event_files(struct tracepoint_path *tps)
                     strcmp(dent->d_name, "ftrace") == 0 ||
                     !system_in_tp_list(dent->d_name, tps))
                         continue;
-                sys = malloc_or_die(strlen(path) + strlen(dent->d_name) + 2);
+                sys = malloc(strlen(path) + strlen(dent->d_name) + 2);
+                if (!sys) {
+                        err = -ENOMEM;
+                        goto out;
+                }
                 sprintf(sys, "%s/%s", path, dent->d_name);
                 ret = stat(sys, &st);
                 if (ret >= 0) {
-                        write_or_die(dent->d_name, strlen(dent->d_name) + 1);
-                        copy_event_system(sys, tps);
+                        ssize_t size = strlen(dent->d_name) + 1;
+
+                        if (write(output_fd, dent->d_name, size) != size ||
+                            copy_event_system(sys, tps) < 0) {
+                                err = -EIO;
+                                free(sys);
+                                goto out;
+                        }
                 }
                 free(sys);
         }
-
+        err = 0;
+out:
         closedir(dir);
         put_tracing_file(path);
+
+        return err;
 }
 
-static void read_proc_kallsyms(void)
+static int read_proc_kallsyms(void)
 {
         unsigned int size;
         const char *path = "/proc/kallsyms";
         struct stat st;
-        int ret;
+        int ret, err = 0;
 
         ret = stat(path, &st);
         if (ret < 0) {
                 /* not found */
                 size = 0;
-                write_or_die(&size, 4);
-                return;
+                if (write(output_fd, &size, 4) != 4)
+                        err = -EIO;
+                return err;
         }
-        record_file(path, 4);
+        return record_file(path, 4);
 }
 
-static void read_ftrace_printk(void)
+static int read_ftrace_printk(void)
 {
         unsigned int size;
         char *path;
         struct stat st;
-        int ret;
+        int ret, err = 0;
 
         path = get_tracing_file("printk_formats");
+        if (!path) {
+                pr_debug("can't get tracing/printk_formats");
+                return -ENOMEM;
+        }
+
         ret = stat(path, &st);
         if (ret < 0) {
                 /* not found */
                 size = 0;
-                write_or_die(&size, 4);
+                if (write(output_fd, &size, 4) != 4)
+                        err = -EIO;
                 goto out;
         }
-        record_file(path, 4);
+        err = record_file(path, 4);
 
 out:
         put_tracing_file(path);
+        return err;
+}
+
+static void
+put_tracepoints_path(struct tracepoint_path *tps)
+{
+        while (tps) {
+                struct tracepoint_path *t = tps;
+
+                tps = tps->next;
+                free(t->name);
+                free(t->system);
+                free(t);
+        }
 }
 
 static struct tracepoint_path *
@@ -396,27 +474,17 @@ get_tracepoints_path(struct list_head *pattrs)
                         continue;
                 ++nr_tracepoints;
                 ppath->next = tracepoint_id_to_path(pos->attr.config);
-                if (!ppath->next)
-                        die("%s\n", "No memory to alloc tracepoints list");
+                if (!ppath->next) {
+                        pr_debug("No memory to alloc tracepoints list\n");
+                        put_tracepoints_path(&path);
+                        return NULL;
+                }
                 ppath = ppath->next;
         }
 
         return nr_tracepoints > 0 ? path.next : NULL;
 }
 
-static void
-put_tracepoints_path(struct tracepoint_path *tps)
-{
-        while (tps) {
-                struct tracepoint_path *t = tps;
-
-                tps = tps->next;
-                free(t->name);
-                free(t->system);
-                free(t);
-        }
-}
-
 bool have_tracepoints(struct list_head *pattrs)
 {
         struct perf_evsel *pos;
@@ -428,9 +496,10 @@ bool have_tracepoints(struct list_head *pattrs)
         return false;
 }
 
-static void tracing_data_header(void)
+static int tracing_data_header(void)
 {
         char buf[20];
+        ssize_t size;
 
         /* just guessing this is someone's birthday.. ;) */
         buf[0] = 23;
@@ -438,9 +507,12 @@ static void tracing_data_header(void)
         buf[2] = 68;
         memcpy(buf + 3, "tracing", 7);
 
-        write_or_die(buf, 10);
+        if (write(output_fd, buf, 10) != 10)
+                return -1;
 
-        write_or_die(VERSION, strlen(VERSION) + 1);
+        size = strlen(VERSION) + 1;
+        if (write(output_fd, VERSION, size) != size)
+                return -1;
 
         /* save endian */
         if (bigendian())
@@ -450,15 +522,19 @@ static void tracing_data_header(void)
 
         read_trace_init(buf[0], buf[0]);
 
-        write_or_die(buf, 1);
+        if (write(output_fd, buf, 1) != 1)
+                return -1;
 
         /* save size of long */
         buf[0] = sizeof(long);
-        write_or_die(buf, 1);
+        if (write(output_fd, buf, 1) != 1)
+                return -1;
 
         /* save page_size */
-        page_size = sysconf(_SC_PAGESIZE);
-        write_or_die(&page_size, 4);
+        if (write(output_fd, &page_size, 4) != 4)
+                return -1;
+
+        return 0;
 }
 
 struct tracing_data *tracing_data_get(struct list_head *pattrs,
@@ -466,6 +542,7 @@ struct tracing_data *tracing_data_get(struct list_head *pattrs,
 {
         struct tracepoint_path *tps;
         struct tracing_data *tdata;
+        int err;
 
         output_fd = fd;
 
@@ -473,7 +550,10 @@ struct tracing_data *tracing_data_get(struct list_head *pattrs,
         if (!tps)
                 return NULL;
 
-        tdata = malloc_or_die(sizeof(*tdata));
+        tdata = malloc(sizeof(*tdata));
+        if (!tdata)
+                return NULL;
+
         tdata->temp = temp;
         tdata->size = 0;
 
@@ -482,12 +562,16 @@ struct tracing_data *tracing_data_get(struct list_head *pattrs,
 
                 snprintf(tdata->temp_file, sizeof(tdata->temp_file),
                          "/tmp/perf-XXXXXX");
-                if (!mkstemp(tdata->temp_file))
-                        die("Can't make temp file");
+                if (!mkstemp(tdata->temp_file)) {
+                        pr_debug("Can't make temp file");
+                        return NULL;
+                }
 
                 temp_fd = open(tdata->temp_file, O_RDWR);
-                if (temp_fd < 0)
-                        die("Can't read '%s'", tdata->temp_file);
+                if (temp_fd < 0) {
+                        pr_debug("Can't read '%s'", tdata->temp_file);
+                        return NULL;
+                }
 
                 /*
                  * Set the temp file the default output, so all the
@@ -496,13 +580,24 @@ struct tracing_data *tracing_data_get(struct list_head *pattrs,
                 output_fd = temp_fd;
         }
 
-        tracing_data_header();
-        read_header_files();
-        read_ftrace_files(tps);
-        read_event_files(tps);
-        read_proc_kallsyms();
-        read_ftrace_printk();
+        err = tracing_data_header();
+        if (err)
+                goto out;
+        err = read_header_files();
+        if (err)
+                goto out;
+        err = read_ftrace_files(tps);
+        if (err)
+                goto out;
+        err = read_event_files(tps);
+        if (err)
+                goto out;
+        err = read_proc_kallsyms();
+        if (err)
+                goto out;
+        err = read_ftrace_printk();
 
+out:
         /*
          * All tracing data are stored by now, we can restore
          * the default output file in case we used temp file.
@@ -513,22 +608,31 @@ struct tracing_data *tracing_data_get(struct list_head *pattrs,
                 output_fd = fd;
         }
 
+        if (err) {
+                free(tdata);
+                tdata = NULL;
+        }
+
         put_tracepoints_path(tps);
         return tdata;
 }
 
-void tracing_data_put(struct tracing_data *tdata)
+int tracing_data_put(struct tracing_data *tdata)
 {
+        int err = 0;
+
         if (tdata->temp) {
-                record_file(tdata->temp_file, 0);
+                err = record_file(tdata->temp_file, 0);
                 unlink(tdata->temp_file);
         }
 
         free(tdata);
+        return err;
 }
 
 int read_tracing_data(int fd, struct list_head *pattrs)
 {
+        int err;
         struct tracing_data *tdata;
 
         /*
@@ -539,6 +643,6 @@ int read_tracing_data(int fd, struct list_head *pattrs)
         if (!tdata)
                 return -ENOMEM;
 
-        tracing_data_put(tdata);
-        return 0;
+        err = tracing_data_put(tdata);
+        return err;
 }
diff --git a/tools/perf/util/trace-event-parse.c b/tools/perf/util/trace-event-parse.c
index 3aabcd687cd5..4454835a9ebc 100644
--- a/tools/perf/util/trace-event-parse.c
+++ b/tools/perf/util/trace-event-parse.c
@@ -183,43 +183,6 @@ void event_format__print(struct event_format *event,
         trace_seq_do_printf(&s);
 }
 
-void print_trace_event(struct pevent *pevent, int cpu, void *data, int size)
-{
-        int type = trace_parse_common_type(pevent, data);
-        struct event_format *event = pevent_find_event(pevent, type);
-
-        if (!event) {
-                warning("ug! no event found for type %d", type);
-                return;
-        }
-
-        event_format__print(event, cpu, data, size);
-}
-
-void print_event(struct pevent *pevent, int cpu, void *data, int size,
-                 unsigned long long nsecs, char *comm)
-{
-        struct pevent_record record;
-        struct trace_seq s;
-        int pid;
-
-        pevent->latency_format = latency_format;
-
-        record.ts = nsecs;
-        record.cpu = cpu;
-        record.size = size;
-        record.data = data;
-        pid = pevent_data_pid(pevent, &record);
-
-        if (!pevent_pid_is_registered(pevent, pid))
-                pevent_register_comm(pevent, comm, pid);
-
-        trace_seq_init(&s);
-        pevent_print_event(pevent, &s, &record);
-        trace_seq_do_printf(&s);
-        printf("\n");
-}
-
 void parse_proc_kallsyms(struct pevent *pevent,
                          char *file, unsigned int size __maybe_unused)
 {
diff --git a/tools/perf/util/trace-event-read.c b/tools/perf/util/trace-event-read.c
index 3741572696af..af215c0d2379 100644
--- a/tools/perf/util/trace-event-read.c
+++ b/tools/perf/util/trace-event-read.c
@@ -18,8 +18,6 @@
  *
  * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  */
-#define _FILE_OFFSET_BITS 64
-
 #include <dirent.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -41,26 +39,14 @@
 
 static int input_fd;
 
-static int read_page;
-
 int file_bigendian;
 int host_bigendian;
 static int long_size;
 
-static ssize_t calc_data_size;
+static ssize_t trace_data_size;
 static bool repipe;
 
-static void *malloc_or_die(int size)
-{
-        void *ret;
-
-        ret = malloc(size);
-        if (!ret)
-                die("malloc");
-        return ret;
-}
-
-static int do_read(int fd, void *buf, int size)
+static int __do_read(int fd, void *buf, int size)
 {
         int rsize = size;
 
@@ -73,8 +59,10 @@ static int do_read(int fd, void *buf, int size)
                 if (repipe) {
                         int retw = write(STDOUT_FILENO, buf, ret);
 
-                        if (retw <= 0 || retw != ret)
-                                die("repiping input file");
+                        if (retw <= 0 || retw != ret) {
+                                pr_debug("repiping input file");
+                                return -1;
+                        }
                 }
 
                 size -= ret;
@@ -84,17 +72,18 @@ static int do_read(int fd, void *buf, int size)
         return rsize;
 }
 
-static int read_or_die(void *data, int size)
+static int do_read(void *data, int size)
 {
         int r;
 
-        r = do_read(input_fd, data, size);
-        if (r <= 0)
-                die("reading input file (size expected=%d received=%d)",
-                    size, r);
+        r = __do_read(input_fd, data, size);
+        if (r <= 0) {
+                pr_debug("reading input file (size expected=%d received=%d)",
+                         size, r);
+                return -1;
+        }
 
-        if (calc_data_size)
-                calc_data_size += r;
+        trace_data_size += r;
 
         return r;
 }
@@ -107,7 +96,7 @@ static void skip(int size)
 
         while (size) {
                 r = size > BUFSIZ ? BUFSIZ : size;
-                read_or_die(buf, r);
+                do_read(buf, r);
                 size -= r;
         };
 }
@@ -116,7 +105,8 @@ static unsigned int read4(struct pevent *pevent)
 {
         unsigned int data;
 
-        read_or_die(&data, 4);
+        if (do_read(&data, 4) < 0)
+                return 0;
         return __data2host4(pevent, data);
 }
 
@@ -124,7 +114,8 @@ static unsigned long long read8(struct pevent *pevent)
 {
         unsigned long long data;
 
-        read_or_die(&data, 8);
+        if (do_read(&data, 8) < 0)
+                return 0;
         return __data2host8(pevent, data);
 }
 
@@ -138,17 +129,23 @@ static char *read_string(void)
 
         for (;;) {
                 r = read(input_fd, &c, 1);
-                if (r < 0)
-                        die("reading input file");
+                if (r < 0) {
+                        pr_debug("reading input file");
+                        goto out;
+                }
 
-                if (!r)
-                        die("no data");
+                if (!r) {
+                        pr_debug("no data");
+                        goto out;
+                }
 
                 if (repipe) {
                         int retw = write(STDOUT_FILENO, &c, 1);
 
-                        if (retw <= 0 || retw != r)
-                                die("repiping input file string");
+                        if (retw <= 0 || retw != r) {
+                                pr_debug("repiping input file string");
+                                goto out;
+                        }
                 }
 
                 buf[size++] = c;
@@ -157,60 +154,79 @@ static char *read_string(void)
                         break;
         }
 
-        if (calc_data_size)
-                calc_data_size += size;
-
-        str = malloc_or_die(size);
-        memcpy(str, buf, size);
+        trace_data_size += size;
 
+        str = malloc(size);
+        if (str)
+                memcpy(str, buf, size);
+out:
         return str;
 }
 
-static void read_proc_kallsyms(struct pevent *pevent)
+static int read_proc_kallsyms(struct pevent *pevent)
 {
         unsigned int size;
         char *buf;
 
         size = read4(pevent);
         if (!size)
-                return;
+                return 0;
+
+        buf = malloc(size + 1);
+        if (buf == NULL)
+                return -1;
 
-        buf = malloc_or_die(size + 1);
-        read_or_die(buf, size);
+        if (do_read(buf, size) < 0) {
+                free(buf);
+                return -1;
+        }
         buf[size] = '\0';
 
         parse_proc_kallsyms(pevent, buf, size);
 
         free(buf);
+        return 0;
 }
 
-static void read_ftrace_printk(struct pevent *pevent)
+static int read_ftrace_printk(struct pevent *pevent)
 {
         unsigned int size;
         char *buf;
 
+        /* it can have 0 size */
         size = read4(pevent);
         if (!size)
-                return;
+                return 0;
+
+        buf = malloc(size);
+        if (buf == NULL)
+                return -1;
 
-        buf = malloc_or_die(size);
-        read_or_die(buf, size);
+        if (do_read(buf, size) < 0) {
+                free(buf);
+                return -1;
+        }
 
         parse_ftrace_printk(pevent, buf, size);
 
         free(buf);
+        return 0;
 }
 
-static void read_header_files(struct pevent *pevent)
+static int read_header_files(struct pevent *pevent)
 {
         unsigned long long size;
         char *header_event;
         char buf[BUFSIZ];
+        int ret = 0;
 
-        read_or_die(buf, 12);
+        if (do_read(buf, 12) < 0)
+                return -1;
 
-        if (memcmp(buf, "header_page", 12) != 0)
-                die("did not read header page");
+        if (memcmp(buf, "header_page", 12) != 0) {
+                pr_debug("did not read header page");
+                return -1;
+        }
 
         size = read8(pevent);
         skip(size);
@@ -221,269 +237,107 @@ static void read_header_files(struct pevent *pevent)
          */
         long_size = header_page_size_size;
 
-        read_or_die(buf, 13);
-        if (memcmp(buf, "header_event", 13) != 0)
-                die("did not read header event");
+        if (do_read(buf, 13) < 0)
+                return -1;
+
+        if (memcmp(buf, "header_event", 13) != 0) {
+                pr_debug("did not read header event");
+                return -1;
+        }
 
         size = read8(pevent);
-        header_event = malloc_or_die(size);
-        read_or_die(header_event, size);
+        header_event = malloc(size);
+        if (header_event == NULL)
+                return -1;
+
+        if (do_read(header_event, size) < 0)
+                ret = -1;
+
         free(header_event);
+        return ret;
 }
 
-static void read_ftrace_file(struct pevent *pevent, unsigned long long size)
+static int read_ftrace_file(struct pevent *pevent, unsigned long long size)
 {
         char *buf;
 
-        buf = malloc_or_die(size);
-        read_or_die(buf, size);
+        buf = malloc(size);
+        if (buf == NULL)
+                return -1;
+
+        if (do_read(buf, size) < 0) {
+                free(buf);
+                return -1;
+        }
+
         parse_ftrace_file(pevent, buf, size);
         free(buf);
+        return 0;
 }
 
-static void read_event_file(struct pevent *pevent, char *sys,
+static int read_event_file(struct pevent *pevent, char *sys,
                             unsigned long long size)
 {
         char *buf;
 
-        buf = malloc_or_die(size);
-        read_or_die(buf, size);
+        buf = malloc(size);
+        if (buf == NULL)
+                return -1;
+
+        if (do_read(buf, size) < 0) {
+                free(buf);
+                return -1;
+        }
+
         parse_event_file(pevent, buf, size, sys);
         free(buf);
+        return 0;
 }
 
-static void read_ftrace_files(struct pevent *pevent)
+static int read_ftrace_files(struct pevent *pevent)
 {
         unsigned long long size;
         int count;
         int i;
+        int ret;
 
         count = read4(pevent);
 
         for (i = 0; i < count; i++) {
                 size = read8(pevent);
-                read_ftrace_file(pevent, size);
+                ret = read_ftrace_file(pevent, size);
+                if (ret)
+                        return ret;
         }
+        return 0;
 }
 
-static void read_event_files(struct pevent *pevent)
+static int read_event_files(struct pevent *pevent)
 {
         unsigned long long size;
         char *sys;
         int systems;
         int count;
         int i,x;
+        int ret;
 
         systems = read4(pevent);
 
         for (i = 0; i < systems; i++) {
                 sys = read_string();
+                if (sys == NULL)
+                        return -1;
 
                 count = read4(pevent);
+
                 for (x=0; x < count; x++) {
                         size = read8(pevent);
-                        read_event_file(pevent, sys, size);
+                        ret = read_event_file(pevent, sys, size);
+                        if (ret)
+                                return ret;
                 }
         }
-}
-
-struct cpu_data {
-        unsigned long long      offset;
-        unsigned long long      size;
-        unsigned long long      timestamp;
-        struct pevent_record    *next;
-        char                    *page;
-        int                     cpu;
-        int                     index;
-        int                     page_size;
-};
-
-static struct cpu_data *cpu_data;
-
-static void update_cpu_data_index(int cpu)
-{
-        cpu_data[cpu].offset += page_size;
-        cpu_data[cpu].size -= page_size;
-        cpu_data[cpu].index = 0;
-}
-
-static void get_next_page(int cpu)
-{
-        off_t save_seek;
-        off_t ret;
-
-        if (!cpu_data[cpu].page)
-                return;
-
-        if (read_page) {
-                if (cpu_data[cpu].size <= page_size) {
-                        free(cpu_data[cpu].page);
-                        cpu_data[cpu].page = NULL;
-                        return;
-                }
-
-                update_cpu_data_index(cpu);
-
-                /* other parts of the code may expect the pointer to not move */
-                save_seek = lseek(input_fd, 0, SEEK_CUR);
-
-                ret = lseek(input_fd, cpu_data[cpu].offset, SEEK_SET);
-                if (ret == (off_t)-1)
-                        die("failed to lseek");
-                ret = read(input_fd, cpu_data[cpu].page, page_size);
-                if (ret < 0)
-                        die("failed to read page");
-
-                /* reset the file pointer back */
-                lseek(input_fd, save_seek, SEEK_SET);
-
-                return;
-        }
-
-        munmap(cpu_data[cpu].page, page_size);
-        cpu_data[cpu].page = NULL;
-
-        if (cpu_data[cpu].size <= page_size)
-                return;
-
-        update_cpu_data_index(cpu);
-
-        cpu_data[cpu].page = mmap(NULL, page_size, PROT_READ, MAP_PRIVATE,
-                                  input_fd, cpu_data[cpu].offset);
-        if (cpu_data[cpu].page == MAP_FAILED)
-                die("failed to mmap cpu %d at offset 0x%llx",
-                    cpu, cpu_data[cpu].offset);
-}
-
-static unsigned int type_len4host(unsigned int type_len_ts)
-{
-        if (file_bigendian)
-                return (type_len_ts >> 27) & ((1 << 5) - 1);
-        else
-                return type_len_ts & ((1 << 5) - 1);
-}
-
-static unsigned int ts4host(unsigned int type_len_ts)
-{
-        if (file_bigendian)
-                return type_len_ts & ((1 << 27) - 1);
-        else
-                return type_len_ts >> 5;
-}
-
-static int calc_index(void *ptr, int cpu)
-{
-        return (unsigned long)ptr - (unsigned long)cpu_data[cpu].page;
-}
-
-struct pevent_record *trace_peek_data(struct pevent *pevent, int cpu)
-{
-        struct pevent_record *data;
-        void *page = cpu_data[cpu].page;
-        int idx = cpu_data[cpu].index;
-        void *ptr = page + idx;
-        unsigned long long extend;
-        unsigned int type_len_ts;
-        unsigned int type_len;
-        unsigned int delta;
-        unsigned int length = 0;
-
-        if (cpu_data[cpu].next)
-                return cpu_data[cpu].next;
-
-        if (!page)
-                return NULL;
-
-        if (!idx) {
-                /* FIXME: handle header page */
-                if (header_page_ts_size != 8)
-                        die("expected a long long type for timestamp");
-                cpu_data[cpu].timestamp = data2host8(pevent, ptr);
-                ptr += 8;
-                switch (header_page_size_size) {
-                case 4:
-                        cpu_data[cpu].page_size = data2host4(pevent, ptr);
-                        ptr += 4;
-                        break;
-                case 8:
-                        cpu_data[cpu].page_size = data2host8(pevent, ptr);
-                        ptr += 8;
-                        break;
-                default:
-                        die("bad long size");
-                }
-                ptr = cpu_data[cpu].page + header_page_data_offset;
-        }
-
-read_again:
-        idx = calc_index(ptr, cpu);
-
-        if (idx >= cpu_data[cpu].page_size) {
-                get_next_page(cpu);
-                return trace_peek_data(pevent, cpu);
-        }
-
-        type_len_ts = data2host4(pevent, ptr);
-        ptr += 4;
-
-        type_len = type_len4host(type_len_ts);
-        delta = ts4host(type_len_ts);
-
-        switch (type_len) {
-        case RINGBUF_TYPE_PADDING:
-                if (!delta)
-                        die("error, hit unexpected end of page");
-                length = data2host4(pevent, ptr);
-                ptr += 4;
-                length *= 4;
-                ptr += length;
-                goto read_again;
-
-        case RINGBUF_TYPE_TIME_EXTEND:
-                extend = data2host4(pevent, ptr);
-                ptr += 4;
-                extend <<= TS_SHIFT;
-                extend += delta;
-                cpu_data[cpu].timestamp += extend;
-                goto read_again;
-
-        case RINGBUF_TYPE_TIME_STAMP:
-                ptr += 12;
-                break;
-        case 0:
-                length = data2host4(pevent, ptr);
-                ptr += 4;
-                die("here! length=%d", length);
-                break;
-        default:
-                length = type_len * 4;
-                break;
-        }
-
-        cpu_data[cpu].timestamp += delta;
-
-        data = malloc_or_die(sizeof(*data));
-        memset(data, 0, sizeof(*data));
-
-        data->ts = cpu_data[cpu].timestamp;
-        data->size = length;
-        data->data = ptr;
-        ptr += length;
-
-        cpu_data[cpu].index = calc_index(ptr, cpu);
-        cpu_data[cpu].next = data;
-
-        return data;
-}
-
-struct pevent_record *trace_read_data(struct pevent *pevent, int cpu)
-{
-        struct pevent_record *data;
-
-        data = trace_peek_data(pevent, cpu);
-        cpu_data[cpu].next = NULL;
-
-        return data;
+        return 0;
 }
 
 ssize_t trace_report(int fd, struct pevent **ppevent, bool __repipe)
@@ -494,58 +348,85 @@ ssize_t trace_report(int fd, struct pevent **ppevent, bool __repipe)
         int show_version = 0;
         int show_funcs = 0;
         int show_printk = 0;
-        ssize_t size;
+        ssize_t size = -1;
+        struct pevent *pevent;
+        int err;
 
-        calc_data_size = 1;
-        repipe = __repipe;
+        *ppevent = NULL;
 
+        repipe = __repipe;
         input_fd = fd;
 
-        read_or_die(buf, 3);
-        if (memcmp(buf, test, 3) != 0)
-                die("no trace data in the file");
+        if (do_read(buf, 3) < 0)
+                return -1;
+        if (memcmp(buf, test, 3) != 0) {
+                pr_debug("no trace data in the file");
+                return -1;
+        }
 
-        read_or_die(buf, 7);
-        if (memcmp(buf, "tracing", 7) != 0)
-                die("not a trace file (missing 'tracing' tag)");
+        if (do_read(buf, 7) < 0)
+                return -1;
+        if (memcmp(buf, "tracing", 7) != 0) {
+                pr_debug("not a trace file (missing 'tracing' tag)");
+                return -1;
+        }
 
         version = read_string();
+        if (version == NULL)
+                return -1;
         if (show_version)
                 printf("version = %s\n", version);
         free(version);
 
-        read_or_die(buf, 1);
+        if (do_read(buf, 1) < 0)
+                return -1;
         file_bigendian = buf[0];
         host_bigendian = bigendian();
 
-        *ppevent = read_trace_init(file_bigendian, host_bigendian);
-        if (*ppevent == NULL)
-                die("read_trace_init failed");
+        pevent = read_trace_init(file_bigendian, host_bigendian);
+        if (pevent == NULL) {
+                pr_debug("read_trace_init failed");
+                goto out;
+        }
 
-        read_or_die(buf, 1);
+        if (do_read(buf, 1) < 0)
+                goto out;
         long_size = buf[0];
 
-        page_size = read4(*ppevent);
-
-        read_header_files(*ppevent);
-
-        read_ftrace_files(*ppevent);
-        read_event_files(*ppevent);
-        read_proc_kallsyms(*ppevent);
-        read_ftrace_printk(*ppevent);
-
-        size = calc_data_size - 1;
-        calc_data_size = 0;
+        page_size = read4(pevent);
+        if (!page_size)
+                goto out;
+
+        err = read_header_files(pevent);
+        if (err)
+                goto out;
+        err = read_ftrace_files(pevent);
+        if (err)
+                goto out;
+        err = read_event_files(pevent);
+        if (err)
+                goto out;
+        err = read_proc_kallsyms(pevent);
+        if (err)
+                goto out;
+        err = read_ftrace_printk(pevent);
+        if (err)
+                goto out;
+
+        size = trace_data_size;
         repipe = false;
 
         if (show_funcs) {
-                pevent_print_funcs(*ppevent);
-                return size;
-        }
-        if (show_printk) {
-                pevent_print_printk(*ppevent);
-                return size;
+                pevent_print_funcs(pevent);
+        } else if (show_printk) {
+                pevent_print_printk(pevent);
         }
 
+        *ppevent = pevent;
+        pevent = NULL;
+
+out:
+        if (pevent)
+                pevent_free(pevent);
         return size;
 }
diff --git a/tools/perf/util/trace-event.h b/tools/perf/util/trace-event.h
index a55fd37ffea1..1978c398ad87 100644
--- a/tools/perf/util/trace-event.h
+++ b/tools/perf/util/trace-event.h
@@ -30,13 +30,9 @@ enum {
 int bigendian(void);
 
 struct pevent *read_trace_init(int file_bigendian, int host_bigendian);
-void print_trace_event(struct pevent *pevent, int cpu, void *data, int size);
 void event_format__print(struct event_format *event,
                          int cpu, void *data, int size);
 
-void print_event(struct pevent *pevent, int cpu, void *data, int size,
-                 unsigned long long nsecs, char *comm);
-
 int parse_ftrace_file(struct pevent *pevent, char *buf, unsigned long size);
 int parse_event_file(struct pevent *pevent,
                      char *buf, unsigned long size, char *sys);
@@ -72,7 +68,7 @@ struct tracing_data {
 
 struct tracing_data *tracing_data_get(struct list_head *pattrs,
                                       int fd, bool temp);
-void tracing_data_put(struct tracing_data *tdata);
+int tracing_data_put(struct tracing_data *tdata);
 
 
 struct addr_location;
diff --git a/tools/perf/util/util.c b/tools/perf/util/util.c
index 805d1f52c5b4..59d868add275 100644
--- a/tools/perf/util/util.c
+++ b/tools/perf/util/util.c
@@ -17,6 +17,8 @@ bool test_attr__enabled;
 bool perf_host  = true;
 bool perf_guest = false;
 
+char tracing_events_path[PATH_MAX + 1] = "/sys/kernel/debug/tracing/events";
+
 void event_attr_init(struct perf_event_attr *attr)
 {
         if (!perf_host)
@@ -242,3 +244,28 @@ void get_term_dimensions(struct winsize *ws)
         ws->ws_row = 25;
         ws->ws_col = 80;
 }
+
+static void set_tracing_events_path(const char *mountpoint)
+{
+        snprintf(tracing_events_path, sizeof(tracing_events_path), "%s/%s",
+                 mountpoint, "tracing/events");
+}
+
+const char *perf_debugfs_mount(const char *mountpoint)
+{
+        const char *mnt;
+
+        mnt = debugfs_mount(mountpoint);
+        if (!mnt)
+                return NULL;
+
+        set_tracing_events_path(mnt);
+
+        return mnt;
+}
+
+void perf_debugfs_set_path(const char *mntpt)
+{
+        snprintf(debugfs_mountpoint, strlen(debugfs_mountpoint), "%s", mntpt);
+        set_tracing_events_path(mntpt);
+}
diff --git a/tools/perf/util/util.h b/tools/perf/util/util.h
index 09b4c26b71aa..a45710b70a55 100644
--- a/tools/perf/util/util.h
+++ b/tools/perf/util/util.h
@@ -1,8 +1,6 @@
 #ifndef GIT_COMPAT_UTIL_H
 #define GIT_COMPAT_UTIL_H
 
-#define _FILE_OFFSET_BITS 64
-
 #ifndef FLEX_ARRAY
 /*
  * See if our compiler is known to support flexible array members.
@@ -73,10 +71,14 @@
 #include <linux/magic.h>
 #include "types.h"
 #include <sys/ttydefaults.h>
+#include <lk/debugfs.h>
 
 extern const char *graph_line;
 extern const char *graph_dotted_line;
 extern char buildid_dir[];
+extern char tracing_events_path[];
+extern void perf_debugfs_set_path(const char *mountpoint);
+const char *perf_debugfs_mount(const char *mountpoint);
 
 /* On most systems <limits.h> would have given us this, but
  * not on some systems (e.g. GNU/Hurd).
@@ -274,5 +276,4 @@ extern unsigned int page_size;
 
 struct winsize;
 void get_term_dimensions(struct winsize *ws);
-
-#endif
+#endif /* GIT_COMPAT_UTIL_H */
diff --git a/tools/scripts/Makefile.include b/tools/scripts/Makefile.include
index 2964b96aa55f..f03e681f8891 100644
--- a/tools/scripts/Makefile.include
+++ b/tools/scripts/Makefile.include
@@ -1,3 +1,4 @@
+ifneq ($(O),)
 ifeq ($(origin O), command line)
         dummy := $(if $(shell test -d $(O) || echo $(O)),$(error O=$(O) does not exist),)
         ABSOLUTE_O := $(shell cd $(O) ; pwd)
@@ -7,9 +8,10 @@ ifeq ($(objtree),)
         objtree := $(O)
 endif
 endif
+endif
 
-ifneq ($(OUTPUT),)
 # check that the output directory actually exists
+ifneq ($(OUTPUT),)
 OUTDIR := $(shell cd $(OUTPUT) && /bin/pwd)
 $(if $(OUTDIR),, $(error output directory "$(OUTPUT)" does not exist))
 endif
@@ -70,7 +72,7 @@ ifndef V
         QUIET_BISON    = @echo '   ' BISON $@;
 
         descend = \
-                @echo '   ' DESCEND $(1); \
+                +@echo '   ' DESCEND $(1); \
                 mkdir -p $(OUTPUT)$(1) && \
                 $(MAKE) $(COMMAND_O) subdir=$(if $(subdir),$(subdir)/$(1),$(1)) $(PRINT_DIR) -C $(1) $(2)
 endif
diff --git a/tools/vm/Makefile b/tools/vm/Makefile
index 8e30e5c40f8a..24e9ddd93fa4 100644
--- a/tools/vm/Makefile
+++ b/tools/vm/Makefile
@@ -1,11 +1,22 @@
 # Makefile for vm tools
+#
+TARGETS=page-types slabinfo
+
+LK_DIR = ../lib/lk
+LIBLK = $(LK_DIR)/liblk.a
 
 CC = $(CROSS_COMPILE)gcc
-CFLAGS = -Wall -Wextra
+CFLAGS = -Wall -Wextra -I../lib/
+LDFLAGS = $(LIBLK)
+
+$(TARGETS): liblk
+
+liblk:
+        make -C $(LK_DIR)
 
-all: page-types slabinfo
 %: %.c
-        $(CC) $(CFLAGS) -o $@ $^
+        $(CC) $(CFLAGS) -o $@ $< $(LDFLAGS)
 
 clean:
         $(RM) page-types slabinfo
+        make -C ../lib/lk clean
diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c
index b76edf2f8333..71c9c2511ee7 100644
--- a/tools/vm/page-types.c
+++ b/tools/vm/page-types.c
@@ -36,7 +36,7 @@
 #include <sys/statfs.h>
 #include "../../include/uapi/linux/magic.h"
 #include "../../include/uapi/linux/kernel-page-flags.h"
-
+#include <lk/debugfs.h>
 
 #ifndef MAX_PATH
 # define MAX_PATH 256
@@ -178,7 +178,7 @@ static int		kpageflags_fd;
 static int              opt_hwpoison;
 static int              opt_unpoison;
 
-static char             hwpoison_debug_fs[MAX_PATH+1];
+static char             *hwpoison_debug_fs;
 static int              hwpoison_inject_fd;
 static int              hwpoison_forget_fd;
 
@@ -458,81 +458,6 @@ static uint64_t kpageflags_flags(uint64_t flags)
         return flags;
 }
 
-/* verify that a mountpoint is actually a debugfs instance */
-static int debugfs_valid_mountpoint(const char *debugfs)
-{
-        struct statfs st_fs;
-
-        if (statfs(debugfs, &st_fs) < 0)
-                return -ENOENT;
-        else if (st_fs.f_type != (long) DEBUGFS_MAGIC)
-                return -ENOENT;
-
-        return 0;
-}
-
-/* find the path to the mounted debugfs */
-static const char *debugfs_find_mountpoint(void)
-{
-        const char *const *ptr;
-        char type[100];
-        FILE *fp;
-
-        ptr = debugfs_known_mountpoints;
-        while (*ptr) {
-                if (debugfs_valid_mountpoint(*ptr) == 0) {
-                        strcpy(hwpoison_debug_fs, *ptr);
-                        return hwpoison_debug_fs;
-                }
-                ptr++;
-        }
-
-        /* give up and parse /proc/mounts */
-        fp = fopen("/proc/mounts", "r");
-        if (fp == NULL)
-                perror("Can't open /proc/mounts for read");
-
-        while (fscanf(fp, "%*s %"
-                      STR(MAX_PATH)
-                      "s %99s %*s %*d %*d\n",
-                      hwpoison_debug_fs, type) == 2) {
-                if (strcmp(type, "debugfs") == 0)
-                        break;
-        }
-        fclose(fp);
-
-        if (strcmp(type, "debugfs") != 0)
-                return NULL;
-
-        return hwpoison_debug_fs;
-}
-
-/* mount the debugfs somewhere if it's not mounted */
-
-static void debugfs_mount(void)
-{
-        const char *const *ptr;
-
-        /* see if it's already mounted */
-        if (debugfs_find_mountpoint())
-                return;
-
-        ptr = debugfs_known_mountpoints;
-        while (*ptr) {
-                if (mount(NULL, *ptr, "debugfs", 0, NULL) == 0) {
-                        /* save the mountpoint */
-                        strcpy(hwpoison_debug_fs, *ptr);
-                        break;
-                }
-                ptr++;
-        }
-
-        if (*ptr == NULL) {
-                perror("mount debugfs");
-                exit(EXIT_FAILURE);
-        }
-}
-
 /*
  * page actions
  */
@@ -541,7 +466,11 @@ static void prepare_hwpoison_fd(void)
 {
         char buf[MAX_PATH + 1];
 
-        debugfs_mount();
+        hwpoison_debug_fs = debugfs_mount(NULL);
+        if (!hwpoison_debug_fs) {
+                perror("mount debugfs");
+                exit(EXIT_FAILURE);
+        }
 
         if (opt_hwpoison && !hwpoison_inject_fd) {
                 snprintf(buf, MAX_PATH, "%s/hwpoison/corrupt-pfn",