aboutsummaryrefslogtreecommitdiffstats
path: root/Documentation/HTML/index.html
diff options
context:
space:
mode:
Diffstat (limited to 'Documentation/HTML/index.html')
-rw-r--r--Documentation/HTML/index.html653
1 files changed, 653 insertions, 0 deletions
diff --git a/Documentation/HTML/index.html b/Documentation/HTML/index.html
new file mode 100644
index 0000000..12c9957
--- /dev/null
+++ b/Documentation/HTML/index.html
@@ -0,0 +1,653 @@
1<html>
2
3<title>KernelShark</title>
4
5<body>
6
7<h1>KernelShark</h1>
8
9<hr><h1><a name="ToC">Table of Contents</a></h1>
10<p><b><a name="ToC_1" href="#Introduction">Introduction</a></b></br>
11<p><b><a name="ToC_2" href="#graph">The Graph View</a></b></br>
12<menu>
13<li><a name="ToC_2_1" href="#graph-zoom-in">Zooming In</a>
14<li><a name="ToC_2_2" href="#graph-zoom-out">Zooming Out</a>
15<li><a name="ToC_2_3" href="#graph-marker">Graph Markers</a>
16<li><a name="ToC_2_4" href="#graph-cursor">Graph Cursor</a>
17<li><a name="ToC_2_5" href="#graph-plots">Graph Plots</a>
18<menu>
19<li><a name="ToC_2_5_1" href="#graph-plot-task">Task Plots</a>
20<li><a name="ToC_2_5_2" href="#graph-plot-cpu">CPU Plots</a>
21</menu>
22</menu>
23<p><b><a name="ToC_3" href="#list">The List View</a></b></br>
24<menu>
25<li><a name="ToC_3_1" href="#list-select">Selecting an event</a>
26<li><a name="ToC_3_2" href="#list-graph-toggle">Graph follows toggle</a>
27</menu>
28<p><b><a name="ToC_4" href="#filters">Filters</a></b></br>
29<menu>
30<li><a name="ToC_4_1" href="#filter-task">Task Filter</a>
31<li><a name="ToC_4_2" href="#filter-event">Event Filter</a>
32<menu>
33<li><a name="ToC_4_2_1" href="#filter-adv-event">Advance Event Filter</a>
34</menu>
35</menu>
36
37
38<h1><a name="Introduction">Introduction</a></h1>
39
40<p>
41KernelShark is a front end reader of <b>trace-cmd(1)</b> output. "trace-cmd record"
42and "trace-cmd extract" create a trace.dat (<b>trace-cmd.dat(5)</b>) file.
43<b>kernelshark</b> can read this file and produce a graph and list view of its data.
44</p>
45
46<img src="images/kshark-open.png">
47
48<p>
49The application has two main viewing areas split by a paned divider. The top half
50is a graphical display of the data and the bottom half is a list view of each
51event. Underneath the menu bar is the graph information area:
52</p>
53
54<h4><a name="graph-info-line">Graph Info Area</a></h4>
55
56<img src="images/kshark-graph-info-line.png">
57
58<p>
59The graph information line displays the timestamp of various locations.
60The Pointer: shows the timestamp of where the mouse pointer is. The Cursor: is
61the timestamp of the cursor location. To select a cursor location, double click
62on the graph. Marker A is set with a left mouse click and Marker B is set
63with a with a left mouse click while holding down the shift key.
64</p>
65
66<p>
67The graph is broken into two parts, the plot title section:
68</p>
69
70<h4><a name="graph-plot-title">Plot Title</a></h4>
71
72<img src="images/kshark-graph-plot-title.png">
73
74<p>
75and the plot area:
76</p>
77
78<img src="images/kshark-graph-plot-area.png">
79
80<p>
81The plot area contains the data of the given plot, where plots can be per CPU or
82per task. The top of the plot area shows a timeline. The numbers in the timeline
83are seconds. The time in the timeline is taken from the timestamps within
84the trace.dat file which are architecture dependent. The time usually is the timestamp
85from when the system started.
86</p>
87
88<p>
89Below the graph is the list area.
90</p>
91
92<h4><a name="list-area">List Area</a></h4>
93
94<img src="images/kshark-list-info-area.png">
95
96<p>
97The list area contains the Page of the list. The list can hold a maximum of
981 million entries per page. If the trace data contains more than a million
99entries, then the list will have more than one page.
100</p>
101
102<p>
103The list area also contains a search field to search for elements in the
104list.
105</p>
106
107<h1><a name="graph">The Graph View</a></h1>
108
109<p>
110The graph view of kernelshark shows graphical plots of the data stored in
111the trace.dat file. The data plots are per CPU or per task.
112When there are too many events within the resolution of the graph,
113the plots will appear as a rainbow colored bar. To make more sense out of
114the graphs, you need to zoom into a given location to see the details of
115that time frame more clearly.
116</p>
117
118<h2><a name="graph-zoom-in">Zooming In</a></h2>
119
120<p>
121To zoom in, left mouse click and hold and then drag the mouse right, release
122to zoom. When you click the left mouse button a line will appear and stay
123at that location. When you move the mouse to the right, another line appears
124and will follow the mouse. When you release the mouse button, the area
125between the two lines become the new width of the screen. That is, the graph
126zooms in until the lines match the width of the screen. This allows you to
127zoom into a specific location.
128</p>
129
130<img src="images/kshark-zoom-in-select.png">
131
132<p>
133The area that you selected will now become the new width of the viewable area.
134The smaller the selection, the deeper the zoom. Note, that you must select 10
135pixels to zoom in. Less than 10 pixels will cancel the zoom. You can continue zooming
136in until you get more details.
137</p>
138
139<img src="images/kshark-zoom-in-3.png">
140
141<p>
142To save on resources, when zooming in, the beginning
143and end of the full trace may not be reachable with the horizontal scroll bar.
144If a plot contains no events within the reachable area, then the line will be empty,
145as CPU 1 is in the above image.
146</p>
147
148<p>
149CPU 0 shows two tasks that were running. One task
150is given a pink/red color and the other a green color. The think colored
151horizontal bar represents a task other than idle was running. The small
152lines that jet out of the bar are where events occur.
153</p>
154
155<p>
156If you zoom in enough such that a single event has enough room between itself
157and other events, the type of event and the name and PID of the task that was running will appear
158over the event.
159</p>
160
161<h2><a name="graph-zoom-out">Zooming Out</a></h2>
162
163<p>
164To zoom back out, left mouse click and hold and then drag the mouse left.
165This time the width between the two lines will become a single pixel.
166The farther apart the lines are, the farther the zoom out will be.
167Zoom out will stop at the width of the full trace.
168</p>
169
170<img src="images/kshark-zoom-out-select.png">
171
172<p>
173When the mouse is over an event, a tool tip will appear showing the event name,
174the <a href="#latency">latency data</a>, the event info, the timestamp and the task
175name and task process ID.
176</p>
177
178<img src="images/kshark-zoom-in-2.png">
179
180<h2><a name="graph-marker">Graph Markers</a></h2>
181
182<p>
183There are two markers that can be placed on the graph as well as a cursor.
184Marker A is set by a left mouse click. When a marker is set, the
185 <a href="#graph-info-line">graph info area</a> will be updated.
186Marker A is represented by a green line:
187</p>
188
189<img src="images/kshark-select-a-1.png">
190
191<p>
192To set Marker B, press and hold the shift key and click the left mouse button.
193Marker B will show up in red.
194</p>
195
196<img src="images/kshark-select-b-1.png">
197
198<p>
199When both the A and B markers are set, the <a href="#graph-info-line">graph info area</a>
200will show the timestamp of where the A and B markers are, as well as the difference
201between the two. All timestamps are in seconds, with a resolution of microseconds.
202</p>
203
204<h2><a name="graph-cursor">Graph Cursor</a></h2>
205
206<p>
207Double clicking on the graph will set the cursor. The cursor is a blue line, and when
208it is set, it will also select the event in the list view that is the closest event at the
209timeline of where the cursor was selected.
210</p>
211
212<img src="images/kshark-cursor-1.png">
213
214<p>
215The above shows that list item 217448 (sys_exit) was the closest event to where
216the cursor was selected.
217</p>
218
219<p>
220Note that setting the cursor with double click will also set Marker A.
221</p>
222
223<h2><a name="graph-plots">Graph Plots</a></h2>
224
225<p>
226The graph data is represented by plots. The data on the plots is either CPU specific or
227task specific. If it is CPU specific, then the data is the timeline of events that
228happened on a given CPU (which CPU is shown in the <a href="#graph-plot-title">plot title area</a>).
229If the plot is task specific, then the timeline of events is for the given
230task regardless of what CPU it was on at the time. The task name is also shown
231in the plot title area.
232</p>
233
234<p>
235By default, all the CPUs within the loaded trace.dat file are plotted.
236There are two ways to plot a task. One way is to right mouse click over a
237displayed task in the graph and select the plot option. This will add the
238task plot directly underneath the CPU plot that the task was on where
239the right mouse click took place. The other way is to use the Plots menu.
240</p>
241
242<img src="images/kshark-plot-menu.png">
243
244<h3><a name="graph-plot-task">Task Plots</a></h3>
245
246<p>
247Selecting the "Tasks" menu item will bring up a dialog with all the tasks
248that were found in the trace data.
249</p>
250
251<img src="images/kshark-plot-task-select.png">
252
253<p>
254Selecting a task in this dialog will add the task plot to the bottom of the graph
255area. Unselecting a task in this dialog will remove the plot.
256</p>
257
258<img src="images/kshark-plot-task-result.png">
259
260<p>
261The colors in the task plots are different depending on which CPU the task
262was on at the time. The CPU plots change colors as different tasks run
263on the CPU, and the task plots change color depending on what CPU the task
264is running on. This makes it easy to see how much a task
265bounces around the CPUs. Zooming in on a task plot also shows some more
266characteristics of the task.
267</p>
268
269<img src="images/kshark-plot-task-zoom-1.png">
270
271<p>
272The hollow green bar that is shown in front of some events in the task
273plot represents when the task was woken up from a sleeping state to
274when it actually ran. The hollow red bar between some events shows that
275the task was preempted by another task even though that task
276was still runnable.
277</p>
278
279<p>
280Since the hollow green bar shows the wake up latency of the task, the
281<a href="#graph-marker">A,B markers</a> can be used to measure that time.
282</p>
283
284<img src="images/kshark-plot-task-measure.png">
285
286<p>
287The above shows that the epiphany-browser with PID 28072 had a 479 microsecond wake up
288latency. The same can be done with the preemption latency.
289</p>
290
291<img src="images/kshark-plot-task-measure-preempt.png">
292
293<h3><a name="graph-plot-cpu">CPU Plots</a></h3>
294
295<p>
296Selecting the "CPUs" plot menu item pops up a dialog that shows the available CPUs that
297can be plotted.
298<p>
299
300<img src="images/kshark-plot-cpu-1.png">
301
302<p>
303Removing a selected CPU and hitting "Apply" will remove that CPU plot.
304</p>
305
306<img src="images/kshark-plot-cpu-2.png">
307
308<p>
309
310<img src="images/kshark-plot-cpu-result.png">
311
312<h1><a name="list">The List View</a></h1>
313
314<p>
315The list view is in the bottom half paned window and can be expanded or shortened
316with the paned handle.
317</p>
318
319<img src="images/kshark-list-adjust.png">
320
321<p>
322The top of the list view contains the <a href="#list-area">list area</a> which has the list page, list
323search, and "graph follows" toggle button. If more than a million events are stored in the
324list, then each set of million will be on a different page.
325</p>
326
327<p>
328The columns of the list are:
329</p>
330
331<ul>
332<li><b>#</b> - the index into the list.
333<li><b>CPU</b> - the CPU that the event occurred on.
334<li><b>Time Stamp</b> - The timestamp of the event. This is in seconds with microsecond
335resolution.
336<li><b>PID</b> - The process ID of the task that was running when the event occurred.
337<li><b><a name="latency">Latency</a></b> - The latency is broken into 5 fields:
338 <ol>
339 <li>Interrupts disabled - 'd' if interrupts are disabled, otherwise '.'
340 <li>Need reschedule - 'N' if the kernel was notified that a schedule is needed, otherwise '.'
341 <li>In IRQ - 'h' if in a hard IRQ (hardware triggered), 's' if in a soft IRQ
342 (context where the kernel initiated a the IRQ handler) or if soft IRQs
343 are disabled, 'H' if in a hard IRQ and soft IRQs are disabled or the hard IRQ
344 triggered while processing a soft IRQ, otherwise '.'
345 <li>Preemption counter - The index of the preemption counter. If it is other
346 than zero, then the kernel will not preempt the running tasks, even
347 if a schedule has been requested by some event. If the counter is zero,
348 then '.' is shown.
349 <li>Lock depth - The depth of the big kernel lock being held. The big kernel
350 lock is recursive (same task may acquire it multiple times). On the first
351 acquisition, the depth is zero. This field will be zero or greater, or
352 '.' if the big kernel lock is not held. When the big kernel lock is
353 finally removed from the kernel, this field will go away as well.
354 </ol>
355<li><b>Event</b> - The name of the event.
356<li><b>Info</b> - The data output of a particular event.
357</ul>
358
359<p>
360The list search can find an event based on the contents in a row. Select a column, a
361match criteria and the content to match to find the next row that matches the
362search. The match criterion is one of the following:
363</p>
364
365<ul>
366<li><b>contains</b> - the row cell of the selected column contains the match data. This works with numbers
367as well.
368<li><b>full match</b> - the row cell of the selected column matches exactly the match data.
369<li><b>does not have</b> - the row cell of the selected column does not contain the match data.
370</ul>
371
372<p>
373The search will find the next event that has the matching data within the column.
374</p>
375
376<h2><a name="list-select">Selecting an event</a></h2>
377<p>
378A single click on a row will select the row, but a double click on a row will select
379that row as well as set the graph cursor to the location of that event. If the plot
380that the event is on is not visible then the graph will adjust its vertical view area
381to show the plot with the selected event. This has no effect on
382<a href="#graph-marker">graph markers</a>.
383<p>
384
385<h2><a name="list-graph-toggle">Graph follows toggle</a></h2>
386
387<p>
388When the "graph follows" toggle is set, then even a single click on a row
389will move the graph cursor. With the mouse focus on the list, using the keyboard
390up and down arrow keys will move the selection of the list as well as the graph
391cursor.
392</p>
393
394<img src="images/kshark-list-graph-follow-1.png">
395<p>
396<img src="images/kshark-list-graph-follow-2.png">
397
398<h1><a name="filters">Filters</a></h1>
399
400<p>
401The amount of data that can be stored in a trace.dat file can be enormous.
402To make any sense out of some traces, it may be required to only display various
403events. The same can be true about tasks.
404Kernelshark has filters for tasks as well as for events. The task filter
405affects both the graph and the list, but the graph and list each have a separate
406event filter.
407<p>
408
409<h2><a name="filter-task">Task Filter</a></h2>
410
411<p>
412The task filter is currently set by a right mouse click over
413an event on either the graph or the list view, and by selecting the option to add or remove the
414task to/from the task filter. The tasks within the task filter are the same for
415both the graph and list, but each can be enabled separately.
416</p>
417
418<p>
419There are two types of task filters:
420</p>
421
422<ul>
423<li>Task Filter - only show tasks that are in this filter
424<li>Hide Tasks - do not display the tasks within this filter
425</ul>
426
427<p>
428If there are any tasks within the Task Filter then only those tasks will be displayed
429when the filter is enabled. Any task within the Hide Tasks filter will not be
430displayed, even if that same task is in the Task Filter.
431</p>
432
433<img src="images/kshark-filter-task-menu.png">
434
435<p>
436When either filter contains a task, the filter can be enabled.
437</p>
438
439<img src="images/kshark-list-enable-filter-1.png">
440
441<p>
442When a task is not in the "Task Filter", the pop up will show the
443menu item "Add task". When a task is in the "Task Filter" the
444pop up will show "Remove task".
445</p>
446
447<p>
448When a task is not in the "Hide Tasks", the pop up will show the
449menu item "Hide task". When a task is in the "Hide Tasks", the
450pop up will show "Show task".
451</p>
452
453<h4>The scheduling events</h4>
454
455<p>
456The events "sched_switch", "sched_wakeup", and "sched_wakeup_new" are treated
457differently by the task filter. Because these events deal with two tasks
458(previous and next, waker and wakee), if either of the tasks should be visible
459then that event is visible regardless if the other task should be hidden.
460This may seem confusing when an event that is hidden shows up in the Task column.
461</p>
462
463<h2><a name="filter-event">Event Filter</a></h2>
464
465<p>
466The graph and list view each have their own event filter. The event filters
467are enabled through the Filter menu on the top menu-bar.
468</p>
469
470<img src="images/kshark-filter.png">
471
472<p>
473Selecting either the "list events" or "graph events" will bring up the event
474dialog with the events that are visible selected.
475</p>
476
477<p>
478Note: these do not mean that the events exist in the trace.dat file. They are
479selected if the events are not to be hidden. Events that do not exist in the trace
480will not be displayed regardless of whether or not they are filtered.
481</p>
482
483<img src="images/kshark-filter-events.png">
484
485<p>
486Clicking on "All" or any of the systems will either deselect all events underneath
487or select all events underneath depending on the previous state of the box being
488selected. By deselecting all events, it makes it easier to enable just a few individual events.
489<p>
490
491<img src="images/kshark-filter-events-sched.png">
492
493<p>
494If it is desired that the graph and list view have the same events filtered, then just
495set up the desired filtering in one and then synchronize the other through
496the filter menu.
497</p>
498
499<img src="images/kshark-filter-sync-graph-1.png">
500
501<ul>
502<li><b>sync graph events with list</b> - will set the graph event filter to be the same
503as what is in the list filter.
504<li><b>sync list events with graph</b> - will set the list event filter to be the same
505as what is in the graph filter.
506</ul>
507
508<h3><a name="filter-adv-event">Advanced Event Filter</a></h3>
509
510<p>
511Filtering on events may not be enough. The ability to filter on the content
512of individual events may be needed. In order to accomplish this, the advanced event
513filtering is used. Selecting the "list advanced event" or "graph advanced event"
514from the Filter menu will pop up the advanced event filtering dialog.
515The graph and list view each have their own advanced event filter.
516</p>
517
518<img src="images/kshark-filter-advance-1.png">
519
520<p>
521The "Filter:" entry at the bottom of the dialog is where the advanced filter is
522written. Above that is helper buttons to pick events, operations and event fields.
523The syntax of the filter is:
524<p>
525
526<pre>
527 FILTER := EVENTS | EVENTS ':' EXPRESSION
528 EVENTS := EVENTS ',' EVENTS | SYSTEM '/' EVENT | SYSTEM | EVENT
529 SYSTEM := any system name
530 EVENT := any event name
531 EXPRESSION := EXPRESSION BOOL EXPRESSION | '(' EXPRESSION ')' | OPERATION
532 BOOL := '&&' | '||'
533 OPERATION := '!' EXPRESSION | LVALUE CMP RVALUE | LVALUE STRCMP STRVALUE
534 CMP := '&gt;' | '&lt;' | '==' | '&gt;=' | '&lt;=' | '!='
535 STRCMP := '==' | '!=' | '=~' | '!~'
536 RVALUE := integer | FIELD
537 STRVALUE := string (double quoted value) | FIELD
538 LVALUE := FIELD | EXPR
539 EXPR := FIELD OP RVALUE | '(' EXPR ')' | EXPR OP EXPR
540 FIELD := a field name of an event
541 OP := '+' | '-' | '*' | '/' | '&lt;&lt;' | '&gt;&gt;' | '&' | '!'
542</pre>
543
544<p>
545Spaces are ignored. The example used in the dialog figure:
546</p>
547
548<pre>
549 sched/sched_switch : next_prio &lt; 100 && (prev_prio &gt; 100 && prev_pid != 0)
550</pre>
551
552<p>
553The <tt>sched/</tt> is not necessary because without it, the filter will process all events
554named <tt>sched_switch</tt>, and since there is only one event
555that has that name, including the <tt>sched/</tt> is redundant.
556<p>
557
558<p>
559The <tt>next_prio</tt>, <tt>prev_prio</tt> and <tt>prev_pid</tt> are all
560event fields of the <tt>sched_swich</tt> event.
561</p>
562
563<p>
564If just <tt>sched</tt> was used and the <tt>/sched_switch</tt> was omitted, it would
565still be a valid filter, but it would behave differently. By just specifying
566a system, the filter will run on all events within that system. When a field
567is encountered that does not belong to an event, then that compare will be set to false.
568</p>
569
570<pre>
571 sched : prev_pid != 0
572 sched : !(prev_pid == 0)
573</pre>
574
575<p>
576The above two filters are not equivalent. They are for the <tt>sched_switch</tt> event,
577but not for the other events. The first filter will return false for all events
578that do not contain the <tt>prev_pid</tt> field, but the second filter would return
579true for all events that do not contain that field. Again, if the event does
580not contain a field, just that compare will be evaluated to false, not the entire
581expression. This means for events that do not have the <tt>prev_pid</tt> field,
582the above filters would be equivalent to:
583</p>
584
585<pre>
586 sched : FALSE
587 sched : !(FALSE)
588</pre>
589
590<p>
591Letting filters contain fields that do not belong to an event be valid
592allows for various tricks where two events can share the same
593filter.
594</p>
595
596<pre>
597 sched_switch, sched_wake.* : next_pid == 1 || pid == 1
598</pre>
599
600<p>
601The schedule events that have <tt>next_pid</tt> and not <tt>pid</tt> as a field
602will just compare the first part of the <tt>||</tt> and those events with
603<tt>pid</tt> but without <tt>next_pid</tt> will be compared against the second
604part of the <tt>||</tt>
605</p>
606
607<p>
608Notice that event names in the filter can be regular expressions.
609</p>
610
611<p>
612String fields can have regular expressions used in their comparing if
613<tt>=~</tt> or <tt>!~</tt> are used.
614</p>
615
616<pre>
617 sched_switch : next_comm =~ "^events/[23]$"
618</pre>
619
620<p>
621The available regular expressions are described in <b>regex(7)</b>.
622</p>
623
624<p>
625Note: When adding an advanced filter, all non-advanced filters
626(added by the event filter dialog box) will be removed, and only the advanced
627filters will stay. But non-advanced filters may be added after advanced
628filters have been. The events that have advanced filters will be shaded
629in the event filter dialog:
630<p>
631
632<img src="images/kshark-filter-event-adv-list.png">
633
634<p>
635Just do not click on the advanced filter box and hit "Apply" unless you want to remove
636the advanced filter. Non-advanced filters can now be selected without affecting
637the advanced filters.
638</p>
639
640<img src="images/kshark-filter-list-adv-irq.png">
641
642<p>
643When advanced filters already exist when the advanced filter dialog box pops up,
644they will be listed in the area at the top of the dialog. Although
645one filter may have been written, the list will be per event. A check box
646is to the left of the filter. When checked, the filter will be deleted if
647the "Apply" button is selected.
648</p>
649
650<img src="images/kshark-filter-del-adv.png">
651
652</body>
653</html>