diff options
author | Changbin Du <changbin.du@intel.com> | 2018-02-17 00:39:37 -0500 |
---|---|---|
committer | Jonathan Corbet <corbet@lwn.net> | 2018-03-07 12:23:06 -0500 |
commit | 8fa4e720e8d919271cdf0da3c0856333246398a4 (patch) | |
tree | 171aeabff773c8398647f0f133ee510ce9fec920 /Documentation | |
parent | b3fdd1f92c1a12b2feda08fcad8ef29a40759bd2 (diff) |
trace doc: convert trace/tracepoint-analysis.txt to rst format
This converts the plain text documentation to reStructuredText format and
add it into Sphinx TOC tree. No essential content change.
Cc: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Changbin Du <changbin.du@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Diffstat (limited to 'Documentation')
-rw-r--r-- | Documentation/trace/index.rst | 1 | ||||
-rw-r--r-- | Documentation/trace/tracepoint-analysis.rst (renamed from Documentation/trace/tracepoint-analysis.txt) | 41 |
2 files changed, 27 insertions, 15 deletions
diff --git a/Documentation/trace/index.rst b/Documentation/trace/index.rst index aa2baad9edf3..61b555192160 100644 --- a/Documentation/trace/index.rst +++ b/Documentation/trace/index.rst | |||
@@ -6,4 +6,5 @@ Linux Tracing Technologies | |||
6 | :maxdepth: 2 | 6 | :maxdepth: 2 |
7 | 7 | ||
8 | ftrace-design | 8 | ftrace-design |
9 | tracepoint-analysis | ||
9 | ftrace-uses | 10 | ftrace-uses |
diff --git a/Documentation/trace/tracepoint-analysis.txt b/Documentation/trace/tracepoint-analysis.rst index 058cc6c9dc56..a4d3ff2e5efb 100644 --- a/Documentation/trace/tracepoint-analysis.txt +++ b/Documentation/trace/tracepoint-analysis.rst | |||
@@ -1,7 +1,7 @@ | |||
1 | Notes on Analysing Behaviour Using Events and Tracepoints | 1 | ========================================================= |
2 | 2 | Notes on Analysing Behaviour Using Events and Tracepoints | |
3 | Documentation written by Mel Gorman | 3 | ========================================================= |
4 | PCL information heavily based on email from Ingo Molnar | 4 | :Author: Mel Gorman (PCL information heavily based on email from Ingo Molnar) |
5 | 5 | ||
6 | 1. Introduction | 6 | 1. Introduction |
7 | =============== | 7 | =============== |
@@ -27,18 +27,18 @@ assumed that the PCL tool tools/perf has been installed and is in your path. | |||
27 | ---------------------- | 27 | ---------------------- |
28 | 28 | ||
29 | All possible events are visible from /sys/kernel/debug/tracing/events. Simply | 29 | All possible events are visible from /sys/kernel/debug/tracing/events. Simply |
30 | calling | 30 | calling:: |
31 | 31 | ||
32 | $ find /sys/kernel/debug/tracing/events -type d | 32 | $ find /sys/kernel/debug/tracing/events -type d |
33 | 33 | ||
34 | will give a fair indication of the number of events available. | 34 | will give a fair indication of the number of events available. |
35 | 35 | ||
36 | 2.2 PCL (Performance Counters for Linux) | 36 | 2.2 PCL (Performance Counters for Linux) |
37 | ------- | 37 | ---------------------------------------- |
38 | 38 | ||
39 | Discovery and enumeration of all counters and events, including tracepoints, | 39 | Discovery and enumeration of all counters and events, including tracepoints, |
40 | are available with the perf tool. Getting a list of available events is a | 40 | are available with the perf tool. Getting a list of available events is a |
41 | simple case of: | 41 | simple case of:: |
42 | 42 | ||
43 | $ perf list 2>&1 | grep Tracepoint | 43 | $ perf list 2>&1 | grep Tracepoint |
44 | ext4:ext4_free_inode [Tracepoint event] | 44 | ext4:ext4_free_inode [Tracepoint event] |
@@ -57,7 +57,7 @@ simple case of: | |||
57 | 57 | ||
58 | See Documentation/trace/events.txt for a proper description on how events | 58 | See Documentation/trace/events.txt for a proper description on how events |
59 | can be enabled system-wide. A short example of enabling all events related | 59 | can be enabled system-wide. A short example of enabling all events related |
60 | to page allocation would look something like: | 60 | to page allocation would look something like:: |
61 | 61 | ||
62 | $ for i in `find /sys/kernel/debug/tracing/events -name "enable" | grep mm_`; do echo 1 > $i; done | 62 | $ for i in `find /sys/kernel/debug/tracing/events -name "enable" | grep mm_`; do echo 1 > $i; done |
63 | 63 | ||
@@ -67,6 +67,7 @@ to page allocation would look something like: | |||
67 | In SystemTap, tracepoints are accessible using the kernel.trace() function | 67 | In SystemTap, tracepoints are accessible using the kernel.trace() function |
68 | call. The following is an example that reports every 5 seconds what processes | 68 | call. The following is an example that reports every 5 seconds what processes |
69 | were allocating the pages. | 69 | were allocating the pages. |
70 | :: | ||
70 | 71 | ||
71 | global page_allocs | 72 | global page_allocs |
72 | 73 | ||
@@ -91,6 +92,7 @@ were allocating the pages. | |||
91 | 92 | ||
92 | By specifying the -a switch and analysing sleep, the system-wide events | 93 | By specifying the -a switch and analysing sleep, the system-wide events |
93 | for a duration of time can be examined. | 94 | for a duration of time can be examined. |
95 | :: | ||
94 | 96 | ||
95 | $ perf stat -a \ | 97 | $ perf stat -a \ |
96 | -e kmem:mm_page_alloc -e kmem:mm_page_free \ | 98 | -e kmem:mm_page_alloc -e kmem:mm_page_free \ |
@@ -118,6 +120,7 @@ basis using set_ftrace_pid. | |||
118 | 120 | ||
119 | Events can be activated and tracked for the duration of a process on a local | 121 | Events can be activated and tracked for the duration of a process on a local |
120 | basis using PCL such as follows. | 122 | basis using PCL such as follows. |
123 | :: | ||
121 | 124 | ||
122 | $ perf stat -e kmem:mm_page_alloc -e kmem:mm_page_free \ | 125 | $ perf stat -e kmem:mm_page_alloc -e kmem:mm_page_free \ |
123 | -e kmem:mm_page_free_batched ./hackbench 10 | 126 | -e kmem:mm_page_free_batched ./hackbench 10 |
@@ -145,6 +148,7 @@ Any workload can exhibit variances between runs and it can be important | |||
145 | to know what the standard deviation is. By and large, this is left to the | 148 | to know what the standard deviation is. By and large, this is left to the |
146 | performance analyst to do it by hand. In the event that the discrete event | 149 | performance analyst to do it by hand. In the event that the discrete event |
147 | occurrences are useful to the performance analyst, then perf can be used. | 150 | occurrences are useful to the performance analyst, then perf can be used. |
151 | :: | ||
148 | 152 | ||
149 | $ perf stat --repeat 5 -e kmem:mm_page_alloc -e kmem:mm_page_free | 153 | $ perf stat --repeat 5 -e kmem:mm_page_alloc -e kmem:mm_page_free |
150 | -e kmem:mm_page_free_batched ./hackbench 10 | 154 | -e kmem:mm_page_free_batched ./hackbench 10 |
@@ -167,6 +171,7 @@ aggregation of discrete events, then a script would need to be developed. | |||
167 | 171 | ||
168 | Using --repeat, it is also possible to view how events are fluctuating over | 172 | Using --repeat, it is also possible to view how events are fluctuating over |
169 | time on a system-wide basis using -a and sleep. | 173 | time on a system-wide basis using -a and sleep. |
174 | :: | ||
170 | 175 | ||
171 | $ perf stat -e kmem:mm_page_alloc -e kmem:mm_page_free \ | 176 | $ perf stat -e kmem:mm_page_alloc -e kmem:mm_page_free \ |
172 | -e kmem:mm_page_free_batched \ | 177 | -e kmem:mm_page_free_batched \ |
@@ -188,9 +193,9 @@ When events are enabled the events that are triggering can be read from | |||
188 | options exist as well. By post-processing the output, further information can | 193 | options exist as well. By post-processing the output, further information can |
189 | be gathered on-line as appropriate. Examples of post-processing might include | 194 | be gathered on-line as appropriate. Examples of post-processing might include |
190 | 195 | ||
191 | o Reading information from /proc for the PID that triggered the event | 196 | - Reading information from /proc for the PID that triggered the event |
192 | o Deriving a higher-level event from a series of lower-level events. | 197 | - Deriving a higher-level event from a series of lower-level events. |
193 | o Calculating latencies between two events | 198 | - Calculating latencies between two events |
194 | 199 | ||
195 | Documentation/trace/postprocess/trace-pagealloc-postprocess.pl is an example | 200 | Documentation/trace/postprocess/trace-pagealloc-postprocess.pl is an example |
196 | script that can read trace_pipe from STDIN or a copy of a trace. When used | 201 | script that can read trace_pipe from STDIN or a copy of a trace. When used |
@@ -200,14 +205,14 @@ and twice to exit. | |||
200 | Simplistically, the script just reads STDIN and counts up events but it | 205 | Simplistically, the script just reads STDIN and counts up events but it |
201 | also can do more such as | 206 | also can do more such as |
202 | 207 | ||
203 | o Derive high-level events from many low-level events. If a number of pages | 208 | - Derive high-level events from many low-level events. If a number of pages |
204 | are freed to the main allocator from the per-CPU lists, it recognises | 209 | are freed to the main allocator from the per-CPU lists, it recognises |
205 | that as one per-CPU drain even though there is no specific tracepoint | 210 | that as one per-CPU drain even though there is no specific tracepoint |
206 | for that event | 211 | for that event |
207 | o It can aggregate based on PID or individual process number | 212 | - It can aggregate based on PID or individual process number |
208 | o In the event memory is getting externally fragmented, it reports | 213 | - In the event memory is getting externally fragmented, it reports |
209 | on whether the fragmentation event was severe or moderate. | 214 | on whether the fragmentation event was severe or moderate. |
210 | o When receiving an event about a PID, it can record who the parent was so | 215 | - When receiving an event about a PID, it can record who the parent was so |
211 | that if large numbers of events are coming from very short-lived | 216 | that if large numbers of events are coming from very short-lived |
212 | processes, the parent process responsible for creating all the helpers | 217 | processes, the parent process responsible for creating all the helpers |
213 | can be identified | 218 | can be identified |
@@ -218,6 +223,7 @@ also can do more such as | |||
218 | There may also be a requirement to identify what functions within a program | 223 | There may also be a requirement to identify what functions within a program |
219 | were generating events within the kernel. To begin this sort of analysis, the | 224 | were generating events within the kernel. To begin this sort of analysis, the |
220 | data must be recorded. At the time of writing, this required root: | 225 | data must be recorded. At the time of writing, this required root: |
226 | :: | ||
221 | 227 | ||
222 | $ perf record -c 1 \ | 228 | $ perf record -c 1 \ |
223 | -e kmem:mm_page_alloc -e kmem:mm_page_free \ | 229 | -e kmem:mm_page_alloc -e kmem:mm_page_free \ |
@@ -232,6 +238,7 @@ very coarse as a result. | |||
232 | 238 | ||
233 | This record outputted a file called perf.data which can be analysed using | 239 | This record outputted a file called perf.data which can be analysed using |
234 | perf report. | 240 | perf report. |
241 | :: | ||
235 | 242 | ||
236 | $ perf report | 243 | $ perf report |
237 | # Samples: 30922 | 244 | # Samples: 30922 |
@@ -258,6 +265,7 @@ within the VDSO. With simple binaries, this will often be the case so let's | |||
258 | take a slightly different example. In the course of writing this, it was | 265 | take a slightly different example. In the course of writing this, it was |
259 | noticed that X was generating an insane amount of page allocations so let's look | 266 | noticed that X was generating an insane amount of page allocations so let's look |
260 | at it: | 267 | at it: |
268 | :: | ||
261 | 269 | ||
262 | $ perf record -c 1 -f \ | 270 | $ perf record -c 1 -f \ |
263 | -e kmem:mm_page_alloc -e kmem:mm_page_free \ | 271 | -e kmem:mm_page_alloc -e kmem:mm_page_free \ |
@@ -265,6 +273,7 @@ at it: | |||
265 | -p `pidof X` | 273 | -p `pidof X` |
266 | 274 | ||
267 | This was interrupted after a few seconds and | 275 | This was interrupted after a few seconds and |
276 | :: | ||
268 | 277 | ||
269 | $ perf report | 278 | $ perf report |
270 | # Samples: 27666 | 279 | # Samples: 27666 |
@@ -282,6 +291,7 @@ This was interrupted after a few seconds and | |||
282 | 291 | ||
283 | So, almost half of the events are occurring in a library. To get an idea which | 292 | So, almost half of the events are occurring in a library. To get an idea which |
284 | symbol: | 293 | symbol: |
294 | :: | ||
285 | 295 | ||
286 | $ perf report --sort comm,dso,symbol | 296 | $ perf report --sort comm,dso,symbol |
287 | # Samples: 27666 | 297 | # Samples: 27666 |
@@ -298,6 +308,7 @@ symbol: | |||
298 | 0.00% Xorg [kernel] [k] ftrace_trace_userstack | 308 | 0.00% Xorg [kernel] [k] ftrace_trace_userstack |
299 | 309 | ||
300 | To see where within the function pixmanFillsse2 things are going wrong: | 310 | To see where within the function pixmanFillsse2 things are going wrong: |
311 | :: | ||
301 | 312 | ||
302 | $ perf annotate pixmanFillsse2 | 313 | $ perf annotate pixmanFillsse2 |
303 | [ ... ] | 314 | [ ... ] |