Merge tag 'trace-v4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace

Pull tracing updates from Steven Rostedt:
 "This release cycle is rather small.  Just a few fixes to tracing.

  The big change is the addition of the hwlat tracer. It not only
  detects SMIs, but also other latency that's caused by the hardware. I
  have detected some latency from large boxes having bus contention"

* tag 'trace-v4.9' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace:
  tracing: Call traceoff trigger after event is recorded
  ftrace/scripts: Add helper script to bisect function tracing problem functions
  tracing: Have max_latency be defined for HWLAT_TRACER as well
  tracing: Add NMI tracing in hwlat detector
  tracing: Have hwlat trace migrate across tracing_cpumask CPUs
  tracing: Add documentation for hwlat_detector tracer
  tracing: Added hardware latency tracer
  ftrace: Access ret_stack->subtime only in the function profiler
  function_graph: Handle TRACE_BPUTS in print_graph_comment
  tracing/uprobe: Drop isdigit() check in create_trace_uprobe

1 files changed, 35 insertions, 0 deletions

diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index ba3326785ca4..2a96b063d659 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -216,6 +216,41 @@ config SCHED_TRACER
           This tracer tracks the latency of the highest priority task
           to be scheduled in, starting from the point it has woken up.
 
+config HWLAT_TRACER
+        bool "Tracer to detect hardware latencies (like SMIs)"
+        select GENERIC_TRACER
+        help
+         This tracer, when enabled will create one or more kernel threads,
+         depening on what the cpumask file is set to, which each thread
+         spinning in a loop looking for interruptions caused by
+         something other than the kernel. For example, if a
+         System Management Interrupt (SMI) takes a noticeable amount of
+         time, this tracer will detect it. This is useful for testing
+         if a system is reliable for Real Time tasks.
+
+         Some files are created in the tracing directory when this
+         is enabled:
+
+           hwlat_detector/width   - time in usecs for how long to spin for
+           hwlat_detector/window  - time in usecs between the start of each
+                                     iteration
+
+         A kernel thread is created that will spin with interrupts disabled
+         for "width" microseconds in every "widow" cycle. It will not spin
+         for "window - width" microseconds, where the system can
+         continue to operate.
+
+         The output will appear in the trace and trace_pipe files.
+
+         When the tracer is not running, it has no affect on the system,
+         but when it is running, it can cause the system to be
+         periodically non responsive. Do not run this tracer on a
+         production system.
+
+         To enable this tracer, echo in "hwlat" into the current_tracer
+         file. Every time a latency is greater than tracing_thresh, it will
+         be recorded into the ring buffer.
+
 config ENABLE_DEFAULT_TRACERS
         bool "Trace process context switches and events"
         depends on !GENERIC_TRACER