diff options
author | Linus Torvalds <torvalds@linux-foundation.org> | 2009-09-17 23:56:37 -0400 |
---|---|---|
committer | Linus Torvalds <torvalds@linux-foundation.org> | 2009-09-17 23:56:37 -0400 |
commit | 1218259b2d09c79ed1113d3a6dbb9a1d6391f5cb (patch) | |
tree | 8f07cd39f6a5f74f41d5be34bc0d843428f04082 | |
parent | ca9a702e50287cf429f1c12832319a26a715e70b (diff) | |
parent | 0efb4d20723d58edbad29d1ff98a86b631adb5e6 (diff) |
Merge branch 'tracing-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
* 'tracing-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip: (44 commits)
vsnprintf: remove duplicate comment of vsnprintf
softirq: add BLOCK_IOPOLL to softirq_to_name
oprofile: fix oprofile regression: select RING_BUFFER_ALLOW_SWAP
tracing: switch function prints from %pf to %ps
vsprintf: add %ps that is the same as %pS but is like %pf
tracing: Fix minor bugs for __unregister_ftrace_function_probe
tracing: remove notrace from __kprobes annotation
tracing: optimize global_trace_clock cachelines
MAINTAINERS: Update tracing tree details
ftrace: document function and function graph implementation
tracing: make testing syscall events a separate configuration
tracing: remove some unused macros
ftrace: add compile-time check on F_printk()
tracing: fix F_printk() typos
tracing: have TRACE_EVENT macro use __flags to not shadow parameter
tracing: add static to generated TRACE_EVENT functions
ring-buffer: typecast cmpxchg to fix PowerPC warning
tracing: add filter event logic to special, mmiotrace and boot tracers
tracing: remove trace_event_types.h
tracing: use the new trace_entries.h to create format files
...
35 files changed, 1506 insertions, 797 deletions
diff --git a/Documentation/trace/events.txt b/Documentation/trace/events.txt index 90e8b3383ba2..78c45a87be57 100644 --- a/Documentation/trace/events.txt +++ b/Documentation/trace/events.txt | |||
@@ -1,7 +1,7 @@ | |||
1 | Event Tracing | 1 | Event Tracing |
2 | 2 | ||
3 | Documentation written by Theodore Ts'o | 3 | Documentation written by Theodore Ts'o |
4 | Updated by Li Zefan | 4 | Updated by Li Zefan and Tom Zanussi |
5 | 5 | ||
6 | 1. Introduction | 6 | 1. Introduction |
7 | =============== | 7 | =============== |
@@ -97,3 +97,185 @@ The format of this boot option is the same as described in section 2.1. | |||
97 | 97 | ||
98 | See The example provided in samples/trace_events | 98 | See The example provided in samples/trace_events |
99 | 99 | ||
100 | 4. Event formats | ||
101 | ================ | ||
102 | |||
103 | Each trace event has a 'format' file associated with it that contains | ||
104 | a description of each field in a logged event. This information can | ||
105 | be used to parse the binary trace stream, and is also the place to | ||
106 | find the field names that can be used in event filters (see section 5). | ||
107 | |||
108 | It also displays the format string that will be used to print the | ||
109 | event in text mode, along with the event name and ID used for | ||
110 | profiling. | ||
111 | |||
112 | Every event has a set of 'common' fields associated with it; these are | ||
113 | the fields prefixed with 'common_'. The other fields vary between | ||
114 | events and correspond to the fields defined in the TRACE_EVENT | ||
115 | definition for that event. | ||
116 | |||
117 | Each field in the format has the form: | ||
118 | |||
119 | field:field-type field-name; offset:N; size:N; | ||
120 | |||
121 | where offset is the offset of the field in the trace record and size | ||
122 | is the size of the data item, in bytes. | ||
123 | |||
124 | For example, here's the information displayed for the 'sched_wakeup' | ||
125 | event: | ||
126 | |||
127 | # cat /debug/tracing/events/sched/sched_wakeup/format | ||
128 | |||
129 | name: sched_wakeup | ||
130 | ID: 60 | ||
131 | format: | ||
132 | field:unsigned short common_type; offset:0; size:2; | ||
133 | field:unsigned char common_flags; offset:2; size:1; | ||
134 | field:unsigned char common_preempt_count; offset:3; size:1; | ||
135 | field:int common_pid; offset:4; size:4; | ||
136 | field:int common_tgid; offset:8; size:4; | ||
137 | |||
138 | field:char comm[TASK_COMM_LEN]; offset:12; size:16; | ||
139 | field:pid_t pid; offset:28; size:4; | ||
140 | field:int prio; offset:32; size:4; | ||
141 | field:int success; offset:36; size:4; | ||
142 | field:int cpu; offset:40; size:4; | ||
143 | |||
144 | print fmt: "task %s:%d [%d] success=%d [%03d]", REC->comm, REC->pid, | ||
145 | REC->prio, REC->success, REC->cpu | ||
146 | |||
147 | This event contains 10 fields, the first 5 common and the remaining 5 | ||
148 | event-specific. All the fields for this event are numeric, except for | ||
149 | 'comm' which is a string, a distinction important for event filtering. | ||
150 | |||
151 | 5. Event filtering | ||
152 | ================== | ||
153 | |||
154 | Trace events can be filtered in the kernel by associating boolean | ||
155 | 'filter expressions' with them. As soon as an event is logged into | ||
156 | the trace buffer, its fields are checked against the filter expression | ||
157 | associated with that event type. An event with field values that | ||
158 | 'match' the filter will appear in the trace output, and an event whose | ||
159 | values don't match will be discarded. An event with no filter | ||
160 | associated with it matches everything, and is the default when no | ||
161 | filter has been set for an event. | ||
162 | |||
163 | 5.1 Expression syntax | ||
164 | --------------------- | ||
165 | |||
166 | A filter expression consists of one or more 'predicates' that can be | ||
167 | combined using the logical operators '&&' and '||'. A predicate is | ||
168 | simply a clause that compares the value of a field contained within a | ||
169 | logged event with a constant value and returns either 0 or 1 depending | ||
170 | on whether the field value matched (1) or didn't match (0): | ||
171 | |||
172 | field-name relational-operator value | ||
173 | |||
174 | Parentheses can be used to provide arbitrary logical groupings and | ||
175 | double-quotes can be used to prevent the shell from interpreting | ||
176 | operators as shell metacharacters. | ||
177 | |||
178 | The field-names available for use in filters can be found in the | ||
179 | 'format' files for trace events (see section 4). | ||
180 | |||
181 | The relational-operators depend on the type of the field being tested: | ||
182 | |||
183 | The operators available for numeric fields are: | ||
184 | |||
185 | ==, !=, <, <=, >, >= | ||
186 | |||
187 | And for string fields they are: | ||
188 | |||
189 | ==, != | ||
190 | |||
191 | Currently, only exact string matches are supported. | ||
192 | |||
193 | Currently, the maximum number of predicates in a filter is 16. | ||
194 | |||
195 | 5.2 Setting filters | ||
196 | ------------------- | ||
197 | |||
198 | A filter for an individual event is set by writing a filter expression | ||
199 | to the 'filter' file for the given event. | ||
200 | |||
201 | For example: | ||
202 | |||
203 | # cd /debug/tracing/events/sched/sched_wakeup | ||
204 | # echo "common_preempt_count > 4" > filter | ||
205 | |||
206 | A slightly more involved example: | ||
207 | |||
208 | # cd /debug/tracing/events/sched/sched_signal_send | ||
209 | # echo "((sig >= 10 && sig < 15) || sig == 17) && comm != bash" > filter | ||
210 | |||
211 | If there is an error in the expression, you'll get an 'Invalid | ||
212 | argument' error when setting it, and the erroneous string along with | ||
213 | an error message can be seen by looking at the filter e.g.: | ||
214 | |||
215 | # cd /debug/tracing/events/sched/sched_signal_send | ||
216 | # echo "((sig >= 10 && sig < 15) || dsig == 17) && comm != bash" > filter | ||
217 | -bash: echo: write error: Invalid argument | ||
218 | # cat filter | ||
219 | ((sig >= 10 && sig < 15) || dsig == 17) && comm != bash | ||
220 | ^ | ||
221 | parse_error: Field not found | ||
222 | |||
223 | Currently the caret ('^') for an error always appears at the beginning of | ||
224 | the filter string; the error message should still be useful though | ||
225 | even without more accurate position info. | ||
226 | |||
227 | 5.3 Clearing filters | ||
228 | -------------------- | ||
229 | |||
230 | To clear the filter for an event, write a '0' to the event's filter | ||
231 | file. | ||
232 | |||
233 | To clear the filters for all events in a subsystem, write a '0' to the | ||
234 | subsystem's filter file. | ||
235 | |||
236 | 5.3 Subsystem filters | ||
237 | --------------------- | ||
238 | |||
239 | For convenience, filters for every event in a subsystem can be set or | ||
240 | cleared as a group by writing a filter expression into the filter file | ||
241 | at the root of the subsytem. Note however, that if a filter for any | ||
242 | event within the subsystem lacks a field specified in the subsystem | ||
243 | filter, or if the filter can't be applied for any other reason, the | ||
244 | filter for that event will retain its previous setting. This can | ||
245 | result in an unintended mixture of filters which could lead to | ||
246 | confusing (to the user who might think different filters are in | ||
247 | effect) trace output. Only filters that reference just the common | ||
248 | fields can be guaranteed to propagate successfully to all events. | ||
249 | |||
250 | Here are a few subsystem filter examples that also illustrate the | ||
251 | above points: | ||
252 | |||
253 | Clear the filters on all events in the sched subsytem: | ||
254 | |||
255 | # cd /sys/kernel/debug/tracing/events/sched | ||
256 | # echo 0 > filter | ||
257 | # cat sched_switch/filter | ||
258 | none | ||
259 | # cat sched_wakeup/filter | ||
260 | none | ||
261 | |||
262 | Set a filter using only common fields for all events in the sched | ||
263 | subsytem (all events end up with the same filter): | ||
264 | |||
265 | # cd /sys/kernel/debug/tracing/events/sched | ||
266 | # echo common_pid == 0 > filter | ||
267 | # cat sched_switch/filter | ||
268 | common_pid == 0 | ||
269 | # cat sched_wakeup/filter | ||
270 | common_pid == 0 | ||
271 | |||
272 | Attempt to set a filter using a non-common field for all events in the | ||
273 | sched subsytem (all events but those that have a prev_pid field retain | ||
274 | their old filters): | ||
275 | |||
276 | # cd /sys/kernel/debug/tracing/events/sched | ||
277 | # echo prev_pid == 0 > filter | ||
278 | # cat sched_switch/filter | ||
279 | prev_pid == 0 | ||
280 | # cat sched_wakeup/filter | ||
281 | common_pid == 0 | ||
diff --git a/Documentation/trace/ftrace-design.txt b/Documentation/trace/ftrace-design.txt new file mode 100644 index 000000000000..7003e10f10f5 --- /dev/null +++ b/Documentation/trace/ftrace-design.txt | |||
@@ -0,0 +1,233 @@ | |||
1 | function tracer guts | ||
2 | ==================== | ||
3 | |||
4 | Introduction | ||
5 | ------------ | ||
6 | |||
7 | Here we will cover the architecture pieces that the common function tracing | ||
8 | code relies on for proper functioning. Things are broken down into increasing | ||
9 | complexity so that you can start simple and at least get basic functionality. | ||
10 | |||
11 | Note that this focuses on architecture implementation details only. If you | ||
12 | want more explanation of a feature in terms of common code, review the common | ||
13 | ftrace.txt file. | ||
14 | |||
15 | |||
16 | Prerequisites | ||
17 | ------------- | ||
18 | |||
19 | Ftrace relies on these features being implemented: | ||
20 | STACKTRACE_SUPPORT - implement save_stack_trace() | ||
21 | TRACE_IRQFLAGS_SUPPORT - implement include/asm/irqflags.h | ||
22 | |||
23 | |||
24 | HAVE_FUNCTION_TRACER | ||
25 | -------------------- | ||
26 | |||
27 | You will need to implement the mcount and the ftrace_stub functions. | ||
28 | |||
29 | The exact mcount symbol name will depend on your toolchain. Some call it | ||
30 | "mcount", "_mcount", or even "__mcount". You can probably figure it out by | ||
31 | running something like: | ||
32 | $ echo 'main(){}' | gcc -x c -S -o - - -pg | grep mcount | ||
33 | call mcount | ||
34 | We'll make the assumption below that the symbol is "mcount" just to keep things | ||
35 | nice and simple in the examples. | ||
36 | |||
37 | Keep in mind that the ABI that is in effect inside of the mcount function is | ||
38 | *highly* architecture/toolchain specific. We cannot help you in this regard, | ||
39 | sorry. Dig up some old documentation and/or find someone more familiar than | ||
40 | you to bang ideas off of. Typically, register usage (argument/scratch/etc...) | ||
41 | is a major issue at this point, especially in relation to the location of the | ||
42 | mcount call (before/after function prologue). You might also want to look at | ||
43 | how glibc has implemented the mcount function for your architecture. It might | ||
44 | be (semi-)relevant. | ||
45 | |||
46 | The mcount function should check the function pointer ftrace_trace_function | ||
47 | to see if it is set to ftrace_stub. If it is, there is nothing for you to do, | ||
48 | so return immediately. If it isn't, then call that function in the same way | ||
49 | the mcount function normally calls __mcount_internal -- the first argument is | ||
50 | the "frompc" while the second argument is the "selfpc" (adjusted to remove the | ||
51 | size of the mcount call that is embedded in the function). | ||
52 | |||
53 | For example, if the function foo() calls bar(), when the bar() function calls | ||
54 | mcount(), the arguments mcount() will pass to the tracer are: | ||
55 | "frompc" - the address bar() will use to return to foo() | ||
56 | "selfpc" - the address bar() (with _mcount() size adjustment) | ||
57 | |||
58 | Also keep in mind that this mcount function will be called *a lot*, so | ||
59 | optimizing for the default case of no tracer will help the smooth running of | ||
60 | your system when tracing is disabled. So the start of the mcount function is | ||
61 | typically the bare min with checking things before returning. That also means | ||
62 | the code flow should usually kept linear (i.e. no branching in the nop case). | ||
63 | This is of course an optimization and not a hard requirement. | ||
64 | |||
65 | Here is some pseudo code that should help (these functions should actually be | ||
66 | implemented in assembly): | ||
67 | |||
68 | void ftrace_stub(void) | ||
69 | { | ||
70 | return; | ||
71 | } | ||
72 | |||
73 | void mcount(void) | ||
74 | { | ||
75 | /* save any bare state needed in order to do initial checking */ | ||
76 | |||
77 | extern void (*ftrace_trace_function)(unsigned long, unsigned long); | ||
78 | if (ftrace_trace_function != ftrace_stub) | ||
79 | goto do_trace; | ||
80 | |||
81 | /* restore any bare state */ | ||
82 | |||
83 | return; | ||
84 | |||
85 | do_trace: | ||
86 | |||
87 | /* save all state needed by the ABI (see paragraph above) */ | ||
88 | |||
89 | unsigned long frompc = ...; | ||
90 | unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; | ||
91 | ftrace_trace_function(frompc, selfpc); | ||
92 | |||
93 | /* restore all state needed by the ABI */ | ||
94 | } | ||
95 | |||
96 | Don't forget to export mcount for modules ! | ||
97 | extern void mcount(void); | ||
98 | EXPORT_SYMBOL(mcount); | ||
99 | |||
100 | |||
101 | HAVE_FUNCTION_TRACE_MCOUNT_TEST | ||
102 | ------------------------------- | ||
103 | |||
104 | This is an optional optimization for the normal case when tracing is turned off | ||
105 | in the system. If you do not enable this Kconfig option, the common ftrace | ||
106 | code will take care of doing the checking for you. | ||
107 | |||
108 | To support this feature, you only need to check the function_trace_stop | ||
109 | variable in the mcount function. If it is non-zero, there is no tracing to be | ||
110 | done at all, so you can return. | ||
111 | |||
112 | This additional pseudo code would simply be: | ||
113 | void mcount(void) | ||
114 | { | ||
115 | /* save any bare state needed in order to do initial checking */ | ||
116 | |||
117 | + if (function_trace_stop) | ||
118 | + return; | ||
119 | |||
120 | extern void (*ftrace_trace_function)(unsigned long, unsigned long); | ||
121 | if (ftrace_trace_function != ftrace_stub) | ||
122 | ... | ||
123 | |||
124 | |||
125 | HAVE_FUNCTION_GRAPH_TRACER | ||
126 | -------------------------- | ||
127 | |||
128 | Deep breath ... time to do some real work. Here you will need to update the | ||
129 | mcount function to check ftrace graph function pointers, as well as implement | ||
130 | some functions to save (hijack) and restore the return address. | ||
131 | |||
132 | The mcount function should check the function pointers ftrace_graph_return | ||
133 | (compare to ftrace_stub) and ftrace_graph_entry (compare to | ||
134 | ftrace_graph_entry_stub). If either of those are not set to the relevant stub | ||
135 | function, call the arch-specific function ftrace_graph_caller which in turn | ||
136 | calls the arch-specific function prepare_ftrace_return. Neither of these | ||
137 | function names are strictly required, but you should use them anyways to stay | ||
138 | consistent across the architecture ports -- easier to compare & contrast | ||
139 | things. | ||
140 | |||
141 | The arguments to prepare_ftrace_return are slightly different than what are | ||
142 | passed to ftrace_trace_function. The second argument "selfpc" is the same, | ||
143 | but the first argument should be a pointer to the "frompc". Typically this is | ||
144 | located on the stack. This allows the function to hijack the return address | ||
145 | temporarily to have it point to the arch-specific function return_to_handler. | ||
146 | That function will simply call the common ftrace_return_to_handler function and | ||
147 | that will return the original return address with which, you can return to the | ||
148 | original call site. | ||
149 | |||
150 | Here is the updated mcount pseudo code: | ||
151 | void mcount(void) | ||
152 | { | ||
153 | ... | ||
154 | if (ftrace_trace_function != ftrace_stub) | ||
155 | goto do_trace; | ||
156 | |||
157 | +#ifdef CONFIG_FUNCTION_GRAPH_TRACER | ||
158 | + extern void (*ftrace_graph_return)(...); | ||
159 | + extern void (*ftrace_graph_entry)(...); | ||
160 | + if (ftrace_graph_return != ftrace_stub || | ||
161 | + ftrace_graph_entry != ftrace_graph_entry_stub) | ||
162 | + ftrace_graph_caller(); | ||
163 | +#endif | ||
164 | |||
165 | /* restore any bare state */ | ||
166 | ... | ||
167 | |||
168 | Here is the pseudo code for the new ftrace_graph_caller assembly function: | ||
169 | #ifdef CONFIG_FUNCTION_GRAPH_TRACER | ||
170 | void ftrace_graph_caller(void) | ||
171 | { | ||
172 | /* save all state needed by the ABI */ | ||
173 | |||
174 | unsigned long *frompc = &...; | ||
175 | unsigned long selfpc = <return address> - MCOUNT_INSN_SIZE; | ||
176 | prepare_ftrace_return(frompc, selfpc); | ||
177 | |||
178 | /* restore all state needed by the ABI */ | ||
179 | } | ||
180 | #endif | ||
181 | |||
182 | For information on how to implement prepare_ftrace_return(), simply look at | ||
183 | the x86 version. The only architecture-specific piece in it is the setup of | ||
184 | the fault recovery table (the asm(...) code). The rest should be the same | ||
185 | across architectures. | ||
186 | |||
187 | Here is the pseudo code for the new return_to_handler assembly function. Note | ||
188 | that the ABI that applies here is different from what applies to the mcount | ||
189 | code. Since you are returning from a function (after the epilogue), you might | ||
190 | be able to skimp on things saved/restored (usually just registers used to pass | ||
191 | return values). | ||
192 | |||
193 | #ifdef CONFIG_FUNCTION_GRAPH_TRACER | ||
194 | void return_to_handler(void) | ||
195 | { | ||
196 | /* save all state needed by the ABI (see paragraph above) */ | ||
197 | |||
198 | void (*original_return_point)(void) = ftrace_return_to_handler(); | ||
199 | |||
200 | /* restore all state needed by the ABI */ | ||
201 | |||
202 | /* this is usually either a return or a jump */ | ||
203 | original_return_point(); | ||
204 | } | ||
205 | #endif | ||
206 | |||
207 | |||
208 | HAVE_FTRACE_NMI_ENTER | ||
209 | --------------------- | ||
210 | |||
211 | If you can't trace NMI functions, then skip this option. | ||
212 | |||
213 | <details to be filled> | ||
214 | |||
215 | |||
216 | HAVE_FTRACE_SYSCALLS | ||
217 | --------------------- | ||
218 | |||
219 | <details to be filled> | ||
220 | |||
221 | |||
222 | HAVE_FTRACE_MCOUNT_RECORD | ||
223 | ------------------------- | ||
224 | |||
225 | See scripts/recordmcount.pl for more info. | ||
226 | |||
227 | <details to be filled> | ||
228 | |||
229 | |||
230 | HAVE_DYNAMIC_FTRACE | ||
231 | --------------------- | ||
232 | |||
233 | <details to be filled> | ||
diff --git a/Documentation/trace/ftrace.txt b/Documentation/trace/ftrace.txt index 355d0f1f8c50..1b6292bbdd6d 100644 --- a/Documentation/trace/ftrace.txt +++ b/Documentation/trace/ftrace.txt | |||
@@ -26,6 +26,12 @@ disabled, and more (ftrace allows for tracer plugins, which | |||
26 | means that the list of tracers can always grow). | 26 | means that the list of tracers can always grow). |
27 | 27 | ||
28 | 28 | ||
29 | Implementation Details | ||
30 | ---------------------- | ||
31 | |||
32 | See ftrace-design.txt for details for arch porters and such. | ||
33 | |||
34 | |||
29 | The File System | 35 | The File System |
30 | --------------- | 36 | --------------- |
31 | 37 | ||
diff --git a/MAINTAINERS b/MAINTAINERS index 7d2e5c05ccdc..e613c6dd709f 100644 --- a/MAINTAINERS +++ b/MAINTAINERS | |||
@@ -2158,13 +2158,16 @@ F: Documentation/filesystems/caching/ | |||
2158 | F: fs/fscache/ | 2158 | F: fs/fscache/ |
2159 | F: include/linux/fscache*.h | 2159 | F: include/linux/fscache*.h |
2160 | 2160 | ||
2161 | FTRACE | 2161 | TRACING |
2162 | M: Steven Rostedt <rostedt@goodmis.org> | 2162 | M: Steven Rostedt <rostedt@goodmis.org> |
2163 | M: Frederic Weisbecker <fweisbec@gmail.com> | ||
2164 | M: Ingo Molnar <mingo@redhat.com> | ||
2165 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git tracing/core | ||
2163 | S: Maintained | 2166 | S: Maintained |
2164 | F: Documentation/trace/ftrace.txt | 2167 | F: Documentation/trace/ftrace.txt |
2165 | F: arch/*/*/*/ftrace.h | 2168 | F: arch/*/*/*/ftrace.h |
2166 | F: arch/*/kernel/ftrace.c | 2169 | F: arch/*/kernel/ftrace.c |
2167 | F: include/*/ftrace.h | 2170 | F: include/*/ftrace.h include/trace/ include/linux/trace*.h |
2168 | F: kernel/trace/ | 2171 | F: kernel/trace/ |
2169 | 2172 | ||
2170 | FUJITSU FR-V (FRV) PORT | 2173 | FUJITSU FR-V (FRV) PORT |
diff --git a/arch/Kconfig b/arch/Kconfig index beea3ccebb5e..7f418bbc261a 100644 --- a/arch/Kconfig +++ b/arch/Kconfig | |||
@@ -9,6 +9,7 @@ config OPROFILE | |||
9 | depends on TRACING_SUPPORT | 9 | depends on TRACING_SUPPORT |
10 | select TRACING | 10 | select TRACING |
11 | select RING_BUFFER | 11 | select RING_BUFFER |
12 | select RING_BUFFER_ALLOW_SWAP | ||
12 | help | 13 | help |
13 | OProfile is a profiling system capable of profiling the | 14 | OProfile is a profiling system capable of profiling the |
14 | whole system, include the kernel, kernel modules, libraries, | 15 | whole system, include the kernel, kernel modules, libraries, |
diff --git a/arch/x86/include/asm/nops.h b/arch/x86/include/asm/nops.h index ad2668ee1aa7..6d8723a766cc 100644 --- a/arch/x86/include/asm/nops.h +++ b/arch/x86/include/asm/nops.h | |||
@@ -65,6 +65,8 @@ | |||
65 | 6: osp nopl 0x00(%eax,%eax,1) | 65 | 6: osp nopl 0x00(%eax,%eax,1) |
66 | 7: nopl 0x00000000(%eax) | 66 | 7: nopl 0x00000000(%eax) |
67 | 8: nopl 0x00000000(%eax,%eax,1) | 67 | 8: nopl 0x00000000(%eax,%eax,1) |
68 | Note: All the above are assumed to be a single instruction. | ||
69 | There is kernel code that depends on this. | ||
68 | */ | 70 | */ |
69 | #define P6_NOP1 GENERIC_NOP1 | 71 | #define P6_NOP1 GENERIC_NOP1 |
70 | #define P6_NOP2 ".byte 0x66,0x90\n" | 72 | #define P6_NOP2 ".byte 0x66,0x90\n" |
diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S index c251be745107..d59fe323807e 100644 --- a/arch/x86/kernel/entry_64.S +++ b/arch/x86/kernel/entry_64.S | |||
@@ -146,7 +146,7 @@ ENTRY(ftrace_graph_caller) | |||
146 | END(ftrace_graph_caller) | 146 | END(ftrace_graph_caller) |
147 | 147 | ||
148 | GLOBAL(return_to_handler) | 148 | GLOBAL(return_to_handler) |
149 | subq $80, %rsp | 149 | subq $24, %rsp |
150 | 150 | ||
151 | /* Save the return values */ | 151 | /* Save the return values */ |
152 | movq %rax, (%rsp) | 152 | movq %rax, (%rsp) |
@@ -155,10 +155,10 @@ GLOBAL(return_to_handler) | |||
155 | 155 | ||
156 | call ftrace_return_to_handler | 156 | call ftrace_return_to_handler |
157 | 157 | ||
158 | movq %rax, 72(%rsp) | 158 | movq %rax, 16(%rsp) |
159 | movq 8(%rsp), %rdx | 159 | movq 8(%rsp), %rdx |
160 | movq (%rsp), %rax | 160 | movq (%rsp), %rax |
161 | addq $72, %rsp | 161 | addq $16, %rsp |
162 | retq | 162 | retq |
163 | #endif | 163 | #endif |
164 | 164 | ||
diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index a43223af98b6..29ca8f53ffbe 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h | |||
@@ -88,7 +88,8 @@ | |||
88 | #endif | 88 | #endif |
89 | 89 | ||
90 | #ifdef CONFIG_FTRACE_MCOUNT_RECORD | 90 | #ifdef CONFIG_FTRACE_MCOUNT_RECORD |
91 | #define MCOUNT_REC() VMLINUX_SYMBOL(__start_mcount_loc) = .; \ | 91 | #define MCOUNT_REC() . = ALIGN(8); \ |
92 | VMLINUX_SYMBOL(__start_mcount_loc) = .; \ | ||
92 | *(__mcount_loc) \ | 93 | *(__mcount_loc) \ |
93 | VMLINUX_SYMBOL(__stop_mcount_loc) = .; | 94 | VMLINUX_SYMBOL(__stop_mcount_loc) = .; |
94 | #else | 95 | #else |
@@ -328,7 +329,6 @@ | |||
328 | /* __*init sections */ \ | 329 | /* __*init sections */ \ |
329 | __init_rodata : AT(ADDR(__init_rodata) - LOAD_OFFSET) { \ | 330 | __init_rodata : AT(ADDR(__init_rodata) - LOAD_OFFSET) { \ |
330 | *(.ref.rodata) \ | 331 | *(.ref.rodata) \ |
331 | MCOUNT_REC() \ | ||
332 | DEV_KEEP(init.rodata) \ | 332 | DEV_KEEP(init.rodata) \ |
333 | DEV_KEEP(exit.rodata) \ | 333 | DEV_KEEP(exit.rodata) \ |
334 | CPU_KEEP(init.rodata) \ | 334 | CPU_KEEP(init.rodata) \ |
@@ -452,6 +452,7 @@ | |||
452 | MEM_DISCARD(init.data) \ | 452 | MEM_DISCARD(init.data) \ |
453 | KERNEL_CTORS() \ | 453 | KERNEL_CTORS() \ |
454 | *(.init.rodata) \ | 454 | *(.init.rodata) \ |
455 | MCOUNT_REC() \ | ||
455 | DEV_DISCARD(init.rodata) \ | 456 | DEV_DISCARD(init.rodata) \ |
456 | CPU_DISCARD(init.rodata) \ | 457 | CPU_DISCARD(init.rodata) \ |
457 | MEM_DISCARD(init.rodata) | 458 | MEM_DISCARD(init.rodata) |
diff --git a/include/linux/ftrace_event.h b/include/linux/ftrace_event.h index 23f7179bf74e..bd099ba82ccc 100644 --- a/include/linux/ftrace_event.h +++ b/include/linux/ftrace_event.h | |||
@@ -1,8 +1,8 @@ | |||
1 | #ifndef _LINUX_FTRACE_EVENT_H | 1 | #ifndef _LINUX_FTRACE_EVENT_H |
2 | #define _LINUX_FTRACE_EVENT_H | 2 | #define _LINUX_FTRACE_EVENT_H |
3 | 3 | ||
4 | #include <linux/trace_seq.h> | ||
5 | #include <linux/ring_buffer.h> | 4 | #include <linux/ring_buffer.h> |
5 | #include <linux/trace_seq.h> | ||
6 | #include <linux/percpu.h> | 6 | #include <linux/percpu.h> |
7 | 7 | ||
8 | struct trace_array; | 8 | struct trace_array; |
@@ -34,7 +34,7 @@ struct trace_entry { | |||
34 | unsigned char flags; | 34 | unsigned char flags; |
35 | unsigned char preempt_count; | 35 | unsigned char preempt_count; |
36 | int pid; | 36 | int pid; |
37 | int tgid; | 37 | int lock_depth; |
38 | }; | 38 | }; |
39 | 39 | ||
40 | #define FTRACE_MAX_EVENT \ | 40 | #define FTRACE_MAX_EVENT \ |
@@ -135,7 +135,7 @@ struct ftrace_event_call { | |||
135 | }; | 135 | }; |
136 | 136 | ||
137 | #define MAX_FILTER_PRED 32 | 137 | #define MAX_FILTER_PRED 32 |
138 | #define MAX_FILTER_STR_VAL 128 | 138 | #define MAX_FILTER_STR_VAL 256 /* Should handle KSYM_SYMBOL_LEN */ |
139 | 139 | ||
140 | extern void destroy_preds(struct ftrace_event_call *call); | 140 | extern void destroy_preds(struct ftrace_event_call *call); |
141 | extern int filter_match_preds(struct ftrace_event_call *call, void *rec); | 141 | extern int filter_match_preds(struct ftrace_event_call *call, void *rec); |
diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h index bcd9c07848be..3a46b7b7abb2 100644 --- a/include/linux/kprobes.h +++ b/include/linux/kprobes.h | |||
@@ -48,13 +48,13 @@ | |||
48 | #define KPROBE_HIT_SSDONE 0x00000008 | 48 | #define KPROBE_HIT_SSDONE 0x00000008 |
49 | 49 | ||
50 | /* Attach to insert probes on any functions which should be ignored*/ | 50 | /* Attach to insert probes on any functions which should be ignored*/ |
51 | #define __kprobes __attribute__((__section__(".kprobes.text"))) notrace | 51 | #define __kprobes __attribute__((__section__(".kprobes.text"))) |
52 | #else /* CONFIG_KPROBES */ | 52 | #else /* CONFIG_KPROBES */ |
53 | typedef int kprobe_opcode_t; | 53 | typedef int kprobe_opcode_t; |
54 | struct arch_specific_insn { | 54 | struct arch_specific_insn { |
55 | int dummy; | 55 | int dummy; |
56 | }; | 56 | }; |
57 | #define __kprobes notrace | 57 | #define __kprobes |
58 | #endif /* CONFIG_KPROBES */ | 58 | #endif /* CONFIG_KPROBES */ |
59 | 59 | ||
60 | struct kprobe; | 60 | struct kprobe; |
diff --git a/include/trace/events/block.h b/include/trace/events/block.h index 9a74b468a229..d86af94691c2 100644 --- a/include/trace/events/block.h +++ b/include/trace/events/block.h | |||
@@ -171,6 +171,7 @@ TRACE_EVENT(block_rq_complete, | |||
171 | (unsigned long long)__entry->sector, | 171 | (unsigned long long)__entry->sector, |
172 | __entry->nr_sector, __entry->errors) | 172 | __entry->nr_sector, __entry->errors) |
173 | ); | 173 | ); |
174 | |||
174 | TRACE_EVENT(block_bio_bounce, | 175 | TRACE_EVENT(block_bio_bounce, |
175 | 176 | ||
176 | TP_PROTO(struct request_queue *q, struct bio *bio), | 177 | TP_PROTO(struct request_queue *q, struct bio *bio), |
@@ -186,7 +187,8 @@ TRACE_EVENT(block_bio_bounce, | |||
186 | ), | 187 | ), |
187 | 188 | ||
188 | TP_fast_assign( | 189 | TP_fast_assign( |
189 | __entry->dev = bio->bi_bdev->bd_dev; | 190 | __entry->dev = bio->bi_bdev ? |
191 | bio->bi_bdev->bd_dev : 0; | ||
190 | __entry->sector = bio->bi_sector; | 192 | __entry->sector = bio->bi_sector; |
191 | __entry->nr_sector = bio->bi_size >> 9; | 193 | __entry->nr_sector = bio->bi_size >> 9; |
192 | blk_fill_rwbs(__entry->rwbs, bio->bi_rw, bio->bi_size); | 194 | blk_fill_rwbs(__entry->rwbs, bio->bi_rw, bio->bi_size); |
diff --git a/include/trace/events/irq.h b/include/trace/events/irq.h index 1cb0c3aa11e6..b89f9db4a404 100644 --- a/include/trace/events/irq.h +++ b/include/trace/events/irq.h | |||
@@ -8,16 +8,17 @@ | |||
8 | #include <linux/interrupt.h> | 8 | #include <linux/interrupt.h> |
9 | 9 | ||
10 | #define softirq_name(sirq) { sirq##_SOFTIRQ, #sirq } | 10 | #define softirq_name(sirq) { sirq##_SOFTIRQ, #sirq } |
11 | #define show_softirq_name(val) \ | 11 | #define show_softirq_name(val) \ |
12 | __print_symbolic(val, \ | 12 | __print_symbolic(val, \ |
13 | softirq_name(HI), \ | 13 | softirq_name(HI), \ |
14 | softirq_name(TIMER), \ | 14 | softirq_name(TIMER), \ |
15 | softirq_name(NET_TX), \ | 15 | softirq_name(NET_TX), \ |
16 | softirq_name(NET_RX), \ | 16 | softirq_name(NET_RX), \ |
17 | softirq_name(BLOCK), \ | 17 | softirq_name(BLOCK), \ |
18 | softirq_name(TASKLET), \ | 18 | softirq_name(BLOCK_IOPOLL), \ |
19 | softirq_name(SCHED), \ | 19 | softirq_name(TASKLET), \ |
20 | softirq_name(HRTIMER), \ | 20 | softirq_name(SCHED), \ |
21 | softirq_name(HRTIMER), \ | ||
21 | softirq_name(RCU)) | 22 | softirq_name(RCU)) |
22 | 23 | ||
23 | /** | 24 | /** |
diff --git a/include/trace/ftrace.h b/include/trace/ftrace.h index 308bafd93325..72a3b437b829 100644 --- a/include/trace/ftrace.h +++ b/include/trace/ftrace.h | |||
@@ -239,9 +239,9 @@ ftrace_format_##call(struct ftrace_event_call *unused, \ | |||
239 | #undef __print_flags | 239 | #undef __print_flags |
240 | #define __print_flags(flag, delim, flag_array...) \ | 240 | #define __print_flags(flag, delim, flag_array...) \ |
241 | ({ \ | 241 | ({ \ |
242 | static const struct trace_print_flags flags[] = \ | 242 | static const struct trace_print_flags __flags[] = \ |
243 | { flag_array, { -1, NULL }}; \ | 243 | { flag_array, { -1, NULL }}; \ |
244 | ftrace_print_flags_seq(p, delim, flag, flags); \ | 244 | ftrace_print_flags_seq(p, delim, flag, __flags); \ |
245 | }) | 245 | }) |
246 | 246 | ||
247 | #undef __print_symbolic | 247 | #undef __print_symbolic |
@@ -254,7 +254,7 @@ ftrace_format_##call(struct ftrace_event_call *unused, \ | |||
254 | 254 | ||
255 | #undef TRACE_EVENT | 255 | #undef TRACE_EVENT |
256 | #define TRACE_EVENT(call, proto, args, tstruct, assign, print) \ | 256 | #define TRACE_EVENT(call, proto, args, tstruct, assign, print) \ |
257 | enum print_line_t \ | 257 | static enum print_line_t \ |
258 | ftrace_raw_output_##call(struct trace_iterator *iter, int flags) \ | 258 | ftrace_raw_output_##call(struct trace_iterator *iter, int flags) \ |
259 | { \ | 259 | { \ |
260 | struct trace_seq *s = &iter->seq; \ | 260 | struct trace_seq *s = &iter->seq; \ |
@@ -317,7 +317,7 @@ ftrace_raw_output_##call(struct trace_iterator *iter, int flags) \ | |||
317 | 317 | ||
318 | #undef TRACE_EVENT | 318 | #undef TRACE_EVENT |
319 | #define TRACE_EVENT(call, proto, args, tstruct, func, print) \ | 319 | #define TRACE_EVENT(call, proto, args, tstruct, func, print) \ |
320 | int \ | 320 | static int \ |
321 | ftrace_define_fields_##call(struct ftrace_event_call *event_call) \ | 321 | ftrace_define_fields_##call(struct ftrace_event_call *event_call) \ |
322 | { \ | 322 | { \ |
323 | struct ftrace_raw_##call field; \ | 323 | struct ftrace_raw_##call field; \ |
diff --git a/kernel/softirq.c b/kernel/softirq.c index 7db25067cd2d..f8749e5216e0 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c | |||
@@ -57,7 +57,7 @@ static struct softirq_action softirq_vec[NR_SOFTIRQS] __cacheline_aligned_in_smp | |||
57 | static DEFINE_PER_CPU(struct task_struct *, ksoftirqd); | 57 | static DEFINE_PER_CPU(struct task_struct *, ksoftirqd); |
58 | 58 | ||
59 | char *softirq_to_name[NR_SOFTIRQS] = { | 59 | char *softirq_to_name[NR_SOFTIRQS] = { |
60 | "HI", "TIMER", "NET_TX", "NET_RX", "BLOCK", | 60 | "HI", "TIMER", "NET_TX", "NET_RX", "BLOCK", "BLOCK_IOPOLL", |
61 | "TASKLET", "SCHED", "HRTIMER", "RCU" | 61 | "TASKLET", "SCHED", "HRTIMER", "RCU" |
62 | }; | 62 | }; |
63 | 63 | ||
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 1ea0d1234f4a..e71634604400 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig | |||
@@ -11,12 +11,18 @@ config NOP_TRACER | |||
11 | 11 | ||
12 | config HAVE_FTRACE_NMI_ENTER | 12 | config HAVE_FTRACE_NMI_ENTER |
13 | bool | 13 | bool |
14 | help | ||
15 | See Documentation/trace/ftrace-implementation.txt | ||
14 | 16 | ||
15 | config HAVE_FUNCTION_TRACER | 17 | config HAVE_FUNCTION_TRACER |
16 | bool | 18 | bool |
19 | help | ||
20 | See Documentation/trace/ftrace-implementation.txt | ||
17 | 21 | ||
18 | config HAVE_FUNCTION_GRAPH_TRACER | 22 | config HAVE_FUNCTION_GRAPH_TRACER |
19 | bool | 23 | bool |
24 | help | ||
25 | See Documentation/trace/ftrace-implementation.txt | ||
20 | 26 | ||
21 | config HAVE_FUNCTION_GRAPH_FP_TEST | 27 | config HAVE_FUNCTION_GRAPH_FP_TEST |
22 | bool | 28 | bool |
@@ -28,21 +34,25 @@ config HAVE_FUNCTION_GRAPH_FP_TEST | |||
28 | config HAVE_FUNCTION_TRACE_MCOUNT_TEST | 34 | config HAVE_FUNCTION_TRACE_MCOUNT_TEST |
29 | bool | 35 | bool |
30 | help | 36 | help |
31 | This gets selected when the arch tests the function_trace_stop | 37 | See Documentation/trace/ftrace-implementation.txt |
32 | variable at the mcount call site. Otherwise, this variable | ||
33 | is tested by the called function. | ||
34 | 38 | ||
35 | config HAVE_DYNAMIC_FTRACE | 39 | config HAVE_DYNAMIC_FTRACE |
36 | bool | 40 | bool |
41 | help | ||
42 | See Documentation/trace/ftrace-implementation.txt | ||
37 | 43 | ||
38 | config HAVE_FTRACE_MCOUNT_RECORD | 44 | config HAVE_FTRACE_MCOUNT_RECORD |
39 | bool | 45 | bool |
46 | help | ||
47 | See Documentation/trace/ftrace-implementation.txt | ||
40 | 48 | ||
41 | config HAVE_HW_BRANCH_TRACER | 49 | config HAVE_HW_BRANCH_TRACER |
42 | bool | 50 | bool |
43 | 51 | ||
44 | config HAVE_SYSCALL_TRACEPOINTS | 52 | config HAVE_SYSCALL_TRACEPOINTS |
45 | bool | 53 | bool |
54 | help | ||
55 | See Documentation/trace/ftrace-implementation.txt | ||
46 | 56 | ||
47 | config TRACER_MAX_TRACE | 57 | config TRACER_MAX_TRACE |
48 | bool | 58 | bool |
@@ -469,6 +479,18 @@ config FTRACE_STARTUP_TEST | |||
469 | functioning properly. It will do tests on all the configured | 479 | functioning properly. It will do tests on all the configured |
470 | tracers of ftrace. | 480 | tracers of ftrace. |
471 | 481 | ||
482 | config EVENT_TRACE_TEST_SYSCALLS | ||
483 | bool "Run selftest on syscall events" | ||
484 | depends on FTRACE_STARTUP_TEST | ||
485 | help | ||
486 | This option will also enable testing every syscall event. | ||
487 | It only enables the event and disables it and runs various loads | ||
488 | with the event enabled. This adds a bit more time for kernel boot | ||
489 | up since it runs this on every system call defined. | ||
490 | |||
491 | TBD - enable a way to actually call the syscalls as we test their | ||
492 | events | ||
493 | |||
472 | config MMIOTRACE | 494 | config MMIOTRACE |
473 | bool "Memory mapped IO tracing" | 495 | bool "Memory mapped IO tracing" |
474 | depends on HAVE_MMIOTRACE_SUPPORT && PCI | 496 | depends on HAVE_MMIOTRACE_SUPPORT && PCI |
diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 8c804e24f96f..cc615f84751b 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c | |||
@@ -1323,11 +1323,10 @@ static int __init ftrace_dyn_table_alloc(unsigned long num_to_init) | |||
1323 | 1323 | ||
1324 | enum { | 1324 | enum { |
1325 | FTRACE_ITER_FILTER = (1 << 0), | 1325 | FTRACE_ITER_FILTER = (1 << 0), |
1326 | FTRACE_ITER_CONT = (1 << 1), | 1326 | FTRACE_ITER_NOTRACE = (1 << 1), |
1327 | FTRACE_ITER_NOTRACE = (1 << 2), | 1327 | FTRACE_ITER_FAILURES = (1 << 2), |
1328 | FTRACE_ITER_FAILURES = (1 << 3), | 1328 | FTRACE_ITER_PRINTALL = (1 << 3), |
1329 | FTRACE_ITER_PRINTALL = (1 << 4), | 1329 | FTRACE_ITER_HASH = (1 << 4), |
1330 | FTRACE_ITER_HASH = (1 << 5), | ||
1331 | }; | 1330 | }; |
1332 | 1331 | ||
1333 | #define FTRACE_BUFF_MAX (KSYM_SYMBOL_LEN+4) /* room for wildcards */ | 1332 | #define FTRACE_BUFF_MAX (KSYM_SYMBOL_LEN+4) /* room for wildcards */ |
@@ -1337,8 +1336,7 @@ struct ftrace_iterator { | |||
1337 | int hidx; | 1336 | int hidx; |
1338 | int idx; | 1337 | int idx; |
1339 | unsigned flags; | 1338 | unsigned flags; |
1340 | unsigned char buffer[FTRACE_BUFF_MAX+1]; | 1339 | struct trace_parser parser; |
1341 | unsigned buffer_idx; | ||
1342 | }; | 1340 | }; |
1343 | 1341 | ||
1344 | static void * | 1342 | static void * |
@@ -1407,7 +1405,7 @@ static int t_hash_show(struct seq_file *m, void *v) | |||
1407 | if (rec->ops->print) | 1405 | if (rec->ops->print) |
1408 | return rec->ops->print(m, rec->ip, rec->ops, rec->data); | 1406 | return rec->ops->print(m, rec->ip, rec->ops, rec->data); |
1409 | 1407 | ||
1410 | seq_printf(m, "%pf:%pf", (void *)rec->ip, (void *)rec->ops->func); | 1408 | seq_printf(m, "%ps:%ps", (void *)rec->ip, (void *)rec->ops->func); |
1411 | 1409 | ||
1412 | if (rec->data) | 1410 | if (rec->data) |
1413 | seq_printf(m, ":%p", rec->data); | 1411 | seq_printf(m, ":%p", rec->data); |
@@ -1517,7 +1515,7 @@ static int t_show(struct seq_file *m, void *v) | |||
1517 | if (!rec) | 1515 | if (!rec) |
1518 | return 0; | 1516 | return 0; |
1519 | 1517 | ||
1520 | seq_printf(m, "%pf\n", (void *)rec->ip); | 1518 | seq_printf(m, "%ps\n", (void *)rec->ip); |
1521 | 1519 | ||
1522 | return 0; | 1520 | return 0; |
1523 | } | 1521 | } |
@@ -1604,6 +1602,11 @@ ftrace_regex_open(struct inode *inode, struct file *file, int enable) | |||
1604 | if (!iter) | 1602 | if (!iter) |
1605 | return -ENOMEM; | 1603 | return -ENOMEM; |
1606 | 1604 | ||
1605 | if (trace_parser_get_init(&iter->parser, FTRACE_BUFF_MAX)) { | ||
1606 | kfree(iter); | ||
1607 | return -ENOMEM; | ||
1608 | } | ||
1609 | |||
1607 | mutex_lock(&ftrace_regex_lock); | 1610 | mutex_lock(&ftrace_regex_lock); |
1608 | if ((file->f_mode & FMODE_WRITE) && | 1611 | if ((file->f_mode & FMODE_WRITE) && |
1609 | (file->f_flags & O_TRUNC)) | 1612 | (file->f_flags & O_TRUNC)) |
@@ -2059,9 +2062,9 @@ __unregister_ftrace_function_probe(char *glob, struct ftrace_probe_ops *ops, | |||
2059 | int i, len = 0; | 2062 | int i, len = 0; |
2060 | char *search; | 2063 | char *search; |
2061 | 2064 | ||
2062 | if (glob && (strcmp(glob, "*") || !strlen(glob))) | 2065 | if (glob && (strcmp(glob, "*") == 0 || !strlen(glob))) |
2063 | glob = NULL; | 2066 | glob = NULL; |
2064 | else { | 2067 | else if (glob) { |
2065 | int not; | 2068 | int not; |
2066 | 2069 | ||
2067 | type = ftrace_setup_glob(glob, strlen(glob), &search, ¬); | 2070 | type = ftrace_setup_glob(glob, strlen(glob), &search, ¬); |
@@ -2196,9 +2199,8 @@ ftrace_regex_write(struct file *file, const char __user *ubuf, | |||
2196 | size_t cnt, loff_t *ppos, int enable) | 2199 | size_t cnt, loff_t *ppos, int enable) |
2197 | { | 2200 | { |
2198 | struct ftrace_iterator *iter; | 2201 | struct ftrace_iterator *iter; |
2199 | char ch; | 2202 | struct trace_parser *parser; |
2200 | size_t read = 0; | 2203 | ssize_t ret, read; |
2201 | ssize_t ret; | ||
2202 | 2204 | ||
2203 | if (!cnt || cnt < 0) | 2205 | if (!cnt || cnt < 0) |
2204 | return 0; | 2206 | return 0; |
@@ -2211,72 +2213,23 @@ ftrace_regex_write(struct file *file, const char __user *ubuf, | |||
2211 | } else | 2213 | } else |
2212 | iter = file->private_data; | 2214 | iter = file->private_data; |
2213 | 2215 | ||
2214 | if (!*ppos) { | 2216 | parser = &iter->parser; |
2215 | iter->flags &= ~FTRACE_ITER_CONT; | 2217 | read = trace_get_user(parser, ubuf, cnt, ppos); |
2216 | iter->buffer_idx = 0; | ||
2217 | } | ||
2218 | |||
2219 | ret = get_user(ch, ubuf++); | ||
2220 | if (ret) | ||
2221 | goto out; | ||
2222 | read++; | ||
2223 | cnt--; | ||
2224 | 2218 | ||
2225 | /* | 2219 | if (trace_parser_loaded(parser) && |
2226 | * If the parser haven't finished with the last write, | 2220 | !trace_parser_cont(parser)) { |
2227 | * continue reading the user input without skipping spaces. | 2221 | ret = ftrace_process_regex(parser->buffer, |
2228 | */ | 2222 | parser->idx, enable); |
2229 | if (!(iter->flags & FTRACE_ITER_CONT)) { | ||
2230 | /* skip white space */ | ||
2231 | while (cnt && isspace(ch)) { | ||
2232 | ret = get_user(ch, ubuf++); | ||
2233 | if (ret) | ||
2234 | goto out; | ||
2235 | read++; | ||
2236 | cnt--; | ||
2237 | } | ||
2238 | |||
2239 | /* only spaces were written */ | ||
2240 | if (isspace(ch)) { | ||
2241 | *ppos += read; | ||
2242 | ret = read; | ||
2243 | goto out; | ||
2244 | } | ||
2245 | |||
2246 | iter->buffer_idx = 0; | ||
2247 | } | ||
2248 | |||
2249 | while (cnt && !isspace(ch)) { | ||
2250 | if (iter->buffer_idx < FTRACE_BUFF_MAX) | ||
2251 | iter->buffer[iter->buffer_idx++] = ch; | ||
2252 | else { | ||
2253 | ret = -EINVAL; | ||
2254 | goto out; | ||
2255 | } | ||
2256 | ret = get_user(ch, ubuf++); | ||
2257 | if (ret) | 2223 | if (ret) |
2258 | goto out; | 2224 | goto out; |
2259 | read++; | ||
2260 | cnt--; | ||
2261 | } | ||
2262 | 2225 | ||
2263 | if (isspace(ch)) { | 2226 | trace_parser_clear(parser); |
2264 | iter->buffer[iter->buffer_idx] = 0; | ||
2265 | ret = ftrace_process_regex(iter->buffer, | ||
2266 | iter->buffer_idx, enable); | ||
2267 | if (ret) | ||
2268 | goto out; | ||
2269 | iter->buffer_idx = 0; | ||
2270 | } else { | ||
2271 | iter->flags |= FTRACE_ITER_CONT; | ||
2272 | iter->buffer[iter->buffer_idx++] = ch; | ||
2273 | } | 2227 | } |
2274 | 2228 | ||
2275 | *ppos += read; | ||
2276 | ret = read; | 2229 | ret = read; |
2277 | out: | ||
2278 | mutex_unlock(&ftrace_regex_lock); | ||
2279 | 2230 | ||
2231 | mutex_unlock(&ftrace_regex_lock); | ||
2232 | out: | ||
2280 | return ret; | 2233 | return ret; |
2281 | } | 2234 | } |
2282 | 2235 | ||
@@ -2381,6 +2334,7 @@ ftrace_regex_release(struct inode *inode, struct file *file, int enable) | |||
2381 | { | 2334 | { |
2382 | struct seq_file *m = (struct seq_file *)file->private_data; | 2335 | struct seq_file *m = (struct seq_file *)file->private_data; |
2383 | struct ftrace_iterator *iter; | 2336 | struct ftrace_iterator *iter; |
2337 | struct trace_parser *parser; | ||
2384 | 2338 | ||
2385 | mutex_lock(&ftrace_regex_lock); | 2339 | mutex_lock(&ftrace_regex_lock); |
2386 | if (file->f_mode & FMODE_READ) { | 2340 | if (file->f_mode & FMODE_READ) { |
@@ -2390,9 +2344,10 @@ ftrace_regex_release(struct inode *inode, struct file *file, int enable) | |||
2390 | } else | 2344 | } else |
2391 | iter = file->private_data; | 2345 | iter = file->private_data; |
2392 | 2346 | ||
2393 | if (iter->buffer_idx) { | 2347 | parser = &iter->parser; |
2394 | iter->buffer[iter->buffer_idx] = 0; | 2348 | if (trace_parser_loaded(parser)) { |
2395 | ftrace_match_records(iter->buffer, iter->buffer_idx, enable); | 2349 | parser->buffer[parser->idx] = 0; |
2350 | ftrace_match_records(parser->buffer, parser->idx, enable); | ||
2396 | } | 2351 | } |
2397 | 2352 | ||
2398 | mutex_lock(&ftrace_lock); | 2353 | mutex_lock(&ftrace_lock); |
@@ -2400,7 +2355,9 @@ ftrace_regex_release(struct inode *inode, struct file *file, int enable) | |||
2400 | ftrace_run_update_code(FTRACE_ENABLE_CALLS); | 2355 | ftrace_run_update_code(FTRACE_ENABLE_CALLS); |
2401 | mutex_unlock(&ftrace_lock); | 2356 | mutex_unlock(&ftrace_lock); |
2402 | 2357 | ||
2358 | trace_parser_put(parser); | ||
2403 | kfree(iter); | 2359 | kfree(iter); |
2360 | |||
2404 | mutex_unlock(&ftrace_regex_lock); | 2361 | mutex_unlock(&ftrace_regex_lock); |
2405 | return 0; | 2362 | return 0; |
2406 | } | 2363 | } |
@@ -2499,7 +2456,7 @@ static int g_show(struct seq_file *m, void *v) | |||
2499 | return 0; | 2456 | return 0; |
2500 | } | 2457 | } |
2501 | 2458 | ||
2502 | seq_printf(m, "%pf\n", v); | 2459 | seq_printf(m, "%ps\n", (void *)*ptr); |
2503 | 2460 | ||
2504 | return 0; | 2461 | return 0; |
2505 | } | 2462 | } |
@@ -2602,12 +2559,10 @@ static ssize_t | |||
2602 | ftrace_graph_write(struct file *file, const char __user *ubuf, | 2559 | ftrace_graph_write(struct file *file, const char __user *ubuf, |
2603 | size_t cnt, loff_t *ppos) | 2560 | size_t cnt, loff_t *ppos) |
2604 | { | 2561 | { |
2605 | unsigned char buffer[FTRACE_BUFF_MAX+1]; | 2562 | struct trace_parser parser; |
2606 | unsigned long *array; | 2563 | unsigned long *array; |
2607 | size_t read = 0; | 2564 | size_t read = 0; |
2608 | ssize_t ret; | 2565 | ssize_t ret; |
2609 | int index = 0; | ||
2610 | char ch; | ||
2611 | 2566 | ||
2612 | if (!cnt || cnt < 0) | 2567 | if (!cnt || cnt < 0) |
2613 | return 0; | 2568 | return 0; |
@@ -2625,51 +2580,26 @@ ftrace_graph_write(struct file *file, const char __user *ubuf, | |||
2625 | } else | 2580 | } else |
2626 | array = file->private_data; | 2581 | array = file->private_data; |
2627 | 2582 | ||
2628 | ret = get_user(ch, ubuf++); | 2583 | if (trace_parser_get_init(&parser, FTRACE_BUFF_MAX)) { |
2629 | if (ret) | 2584 | ret = -ENOMEM; |
2630 | goto out; | 2585 | goto out; |
2631 | read++; | ||
2632 | cnt--; | ||
2633 | |||
2634 | /* skip white space */ | ||
2635 | while (cnt && isspace(ch)) { | ||
2636 | ret = get_user(ch, ubuf++); | ||
2637 | if (ret) | ||
2638 | goto out; | ||
2639 | read++; | ||
2640 | cnt--; | ||
2641 | } | 2586 | } |
2642 | 2587 | ||
2643 | if (isspace(ch)) { | 2588 | read = trace_get_user(&parser, ubuf, cnt, ppos); |
2644 | *ppos += read; | ||
2645 | ret = read; | ||
2646 | goto out; | ||
2647 | } | ||
2648 | 2589 | ||
2649 | while (cnt && !isspace(ch)) { | 2590 | if (trace_parser_loaded((&parser))) { |
2650 | if (index < FTRACE_BUFF_MAX) | 2591 | parser.buffer[parser.idx] = 0; |
2651 | buffer[index++] = ch; | 2592 | |
2652 | else { | 2593 | /* we allow only one expression at a time */ |
2653 | ret = -EINVAL; | 2594 | ret = ftrace_set_func(array, &ftrace_graph_count, |
2654 | goto out; | 2595 | parser.buffer); |
2655 | } | ||
2656 | ret = get_user(ch, ubuf++); | ||
2657 | if (ret) | 2596 | if (ret) |
2658 | goto out; | 2597 | goto out; |
2659 | read++; | ||
2660 | cnt--; | ||
2661 | } | 2598 | } |
2662 | buffer[index] = 0; | ||
2663 | |||
2664 | /* we allow only one expression at a time */ | ||
2665 | ret = ftrace_set_func(array, &ftrace_graph_count, buffer); | ||
2666 | if (ret) | ||
2667 | goto out; | ||
2668 | |||
2669 | file->f_pos += read; | ||
2670 | 2599 | ||
2671 | ret = read; | 2600 | ret = read; |
2672 | out: | 2601 | out: |
2602 | trace_parser_put(&parser); | ||
2673 | mutex_unlock(&graph_lock); | 2603 | mutex_unlock(&graph_lock); |
2674 | 2604 | ||
2675 | return ret; | 2605 | return ret; |
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 454e74e718cf..6eef38923b07 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c | |||
@@ -701,8 +701,8 @@ static int rb_head_page_set(struct ring_buffer_per_cpu *cpu_buffer, | |||
701 | 701 | ||
702 | val &= ~RB_FLAG_MASK; | 702 | val &= ~RB_FLAG_MASK; |
703 | 703 | ||
704 | ret = (unsigned long)cmpxchg(&list->next, | 704 | ret = cmpxchg((unsigned long *)&list->next, |
705 | val | old_flag, val | new_flag); | 705 | val | old_flag, val | new_flag); |
706 | 706 | ||
707 | /* check if the reader took the page */ | 707 | /* check if the reader took the page */ |
708 | if ((ret & ~RB_FLAG_MASK) != val) | 708 | if ((ret & ~RB_FLAG_MASK) != val) |
@@ -794,7 +794,7 @@ static int rb_head_page_replace(struct buffer_page *old, | |||
794 | val = *ptr & ~RB_FLAG_MASK; | 794 | val = *ptr & ~RB_FLAG_MASK; |
795 | val |= RB_PAGE_HEAD; | 795 | val |= RB_PAGE_HEAD; |
796 | 796 | ||
797 | ret = cmpxchg(ptr, val, &new->list); | 797 | ret = cmpxchg(ptr, val, (unsigned long)&new->list); |
798 | 798 | ||
799 | return ret == val; | 799 | return ret == val; |
800 | } | 800 | } |
@@ -2997,15 +2997,12 @@ static void rb_advance_iter(struct ring_buffer_iter *iter) | |||
2997 | } | 2997 | } |
2998 | 2998 | ||
2999 | static struct ring_buffer_event * | 2999 | static struct ring_buffer_event * |
3000 | rb_buffer_peek(struct ring_buffer *buffer, int cpu, u64 *ts) | 3000 | rb_buffer_peek(struct ring_buffer_per_cpu *cpu_buffer, u64 *ts) |
3001 | { | 3001 | { |
3002 | struct ring_buffer_per_cpu *cpu_buffer; | ||
3003 | struct ring_buffer_event *event; | 3002 | struct ring_buffer_event *event; |
3004 | struct buffer_page *reader; | 3003 | struct buffer_page *reader; |
3005 | int nr_loops = 0; | 3004 | int nr_loops = 0; |
3006 | 3005 | ||
3007 | cpu_buffer = buffer->buffers[cpu]; | ||
3008 | |||
3009 | again: | 3006 | again: |
3010 | /* | 3007 | /* |
3011 | * We repeat when a timestamp is encountered. It is possible | 3008 | * We repeat when a timestamp is encountered. It is possible |
@@ -3049,7 +3046,7 @@ rb_buffer_peek(struct ring_buffer *buffer, int cpu, u64 *ts) | |||
3049 | case RINGBUF_TYPE_DATA: | 3046 | case RINGBUF_TYPE_DATA: |
3050 | if (ts) { | 3047 | if (ts) { |
3051 | *ts = cpu_buffer->read_stamp + event->time_delta; | 3048 | *ts = cpu_buffer->read_stamp + event->time_delta; |
3052 | ring_buffer_normalize_time_stamp(buffer, | 3049 | ring_buffer_normalize_time_stamp(cpu_buffer->buffer, |
3053 | cpu_buffer->cpu, ts); | 3050 | cpu_buffer->cpu, ts); |
3054 | } | 3051 | } |
3055 | return event; | 3052 | return event; |
@@ -3168,7 +3165,7 @@ ring_buffer_peek(struct ring_buffer *buffer, int cpu, u64 *ts) | |||
3168 | local_irq_save(flags); | 3165 | local_irq_save(flags); |
3169 | if (dolock) | 3166 | if (dolock) |
3170 | spin_lock(&cpu_buffer->reader_lock); | 3167 | spin_lock(&cpu_buffer->reader_lock); |
3171 | event = rb_buffer_peek(buffer, cpu, ts); | 3168 | event = rb_buffer_peek(cpu_buffer, ts); |
3172 | if (event && event->type_len == RINGBUF_TYPE_PADDING) | 3169 | if (event && event->type_len == RINGBUF_TYPE_PADDING) |
3173 | rb_advance_reader(cpu_buffer); | 3170 | rb_advance_reader(cpu_buffer); |
3174 | if (dolock) | 3171 | if (dolock) |
@@ -3237,7 +3234,7 @@ ring_buffer_consume(struct ring_buffer *buffer, int cpu, u64 *ts) | |||
3237 | if (dolock) | 3234 | if (dolock) |
3238 | spin_lock(&cpu_buffer->reader_lock); | 3235 | spin_lock(&cpu_buffer->reader_lock); |
3239 | 3236 | ||
3240 | event = rb_buffer_peek(buffer, cpu, ts); | 3237 | event = rb_buffer_peek(cpu_buffer, ts); |
3241 | if (event) | 3238 | if (event) |
3242 | rb_advance_reader(cpu_buffer); | 3239 | rb_advance_reader(cpu_buffer); |
3243 | 3240 | ||
diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 5c75deeefe30..fd52a19dd172 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c | |||
@@ -339,6 +339,112 @@ static struct { | |||
339 | 339 | ||
340 | int trace_clock_id; | 340 | int trace_clock_id; |
341 | 341 | ||
342 | /* | ||
343 | * trace_parser_get_init - gets the buffer for trace parser | ||
344 | */ | ||
345 | int trace_parser_get_init(struct trace_parser *parser, int size) | ||
346 | { | ||
347 | memset(parser, 0, sizeof(*parser)); | ||
348 | |||
349 | parser->buffer = kmalloc(size, GFP_KERNEL); | ||
350 | if (!parser->buffer) | ||
351 | return 1; | ||
352 | |||
353 | parser->size = size; | ||
354 | return 0; | ||
355 | } | ||
356 | |||
357 | /* | ||
358 | * trace_parser_put - frees the buffer for trace parser | ||
359 | */ | ||
360 | void trace_parser_put(struct trace_parser *parser) | ||
361 | { | ||
362 | kfree(parser->buffer); | ||
363 | } | ||
364 | |||
365 | /* | ||
366 | * trace_get_user - reads the user input string separated by space | ||
367 | * (matched by isspace(ch)) | ||
368 | * | ||
369 | * For each string found the 'struct trace_parser' is updated, | ||
370 | * and the function returns. | ||
371 | * | ||
372 | * Returns number of bytes read. | ||
373 | * | ||
374 | * See kernel/trace/trace.h for 'struct trace_parser' details. | ||
375 | */ | ||
376 | int trace_get_user(struct trace_parser *parser, const char __user *ubuf, | ||
377 | size_t cnt, loff_t *ppos) | ||
378 | { | ||
379 | char ch; | ||
380 | size_t read = 0; | ||
381 | ssize_t ret; | ||
382 | |||
383 | if (!*ppos) | ||
384 | trace_parser_clear(parser); | ||
385 | |||
386 | ret = get_user(ch, ubuf++); | ||
387 | if (ret) | ||
388 | goto out; | ||
389 | |||
390 | read++; | ||
391 | cnt--; | ||
392 | |||
393 | /* | ||
394 | * The parser is not finished with the last write, | ||
395 | * continue reading the user input without skipping spaces. | ||
396 | */ | ||
397 | if (!parser->cont) { | ||
398 | /* skip white space */ | ||
399 | while (cnt && isspace(ch)) { | ||
400 | ret = get_user(ch, ubuf++); | ||
401 | if (ret) | ||
402 | goto out; | ||
403 | read++; | ||
404 | cnt--; | ||
405 | } | ||
406 | |||
407 | /* only spaces were written */ | ||
408 | if (isspace(ch)) { | ||
409 | *ppos += read; | ||
410 | ret = read; | ||
411 | goto out; | ||
412 | } | ||
413 | |||
414 | parser->idx = 0; | ||
415 | } | ||
416 | |||
417 | /* read the non-space input */ | ||
418 | while (cnt && !isspace(ch)) { | ||
419 | if (parser->idx < parser->size) | ||
420 | parser->buffer[parser->idx++] = ch; | ||
421 | else { | ||
422 | ret = -EINVAL; | ||
423 | goto out; | ||
424 | } | ||
425 | ret = get_user(ch, ubuf++); | ||
426 | if (ret) | ||
427 | goto out; | ||
428 | read++; | ||
429 | cnt--; | ||
430 | } | ||
431 | |||
432 | /* We either got finished input or we have to wait for another call. */ | ||
433 | if (isspace(ch)) { | ||
434 | parser->buffer[parser->idx] = 0; | ||
435 | parser->cont = false; | ||
436 | } else { | ||
437 | parser->cont = true; | ||
438 | parser->buffer[parser->idx++] = ch; | ||
439 | } | ||
440 | |||
441 | *ppos += read; | ||
442 | ret = read; | ||
443 | |||
444 | out: | ||
445 | return ret; | ||
446 | } | ||
447 | |||
342 | ssize_t trace_seq_to_user(struct trace_seq *s, char __user *ubuf, size_t cnt) | 448 | ssize_t trace_seq_to_user(struct trace_seq *s, char __user *ubuf, size_t cnt) |
343 | { | 449 | { |
344 | int len; | 450 | int len; |
@@ -719,6 +825,11 @@ static void trace_init_cmdlines(void) | |||
719 | cmdline_idx = 0; | 825 | cmdline_idx = 0; |
720 | } | 826 | } |
721 | 827 | ||
828 | int is_tracing_stopped(void) | ||
829 | { | ||
830 | return trace_stop_count; | ||
831 | } | ||
832 | |||
722 | /** | 833 | /** |
723 | * ftrace_off_permanent - disable all ftrace code permanently | 834 | * ftrace_off_permanent - disable all ftrace code permanently |
724 | * | 835 | * |
@@ -886,7 +997,7 @@ tracing_generic_entry_update(struct trace_entry *entry, unsigned long flags, | |||
886 | 997 | ||
887 | entry->preempt_count = pc & 0xff; | 998 | entry->preempt_count = pc & 0xff; |
888 | entry->pid = (tsk) ? tsk->pid : 0; | 999 | entry->pid = (tsk) ? tsk->pid : 0; |
889 | entry->tgid = (tsk) ? tsk->tgid : 0; | 1000 | entry->lock_depth = (tsk) ? tsk->lock_depth : 0; |
890 | entry->flags = | 1001 | entry->flags = |
891 | #ifdef CONFIG_TRACE_IRQFLAGS_SUPPORT | 1002 | #ifdef CONFIG_TRACE_IRQFLAGS_SUPPORT |
892 | (irqs_disabled_flags(flags) ? TRACE_FLAG_IRQS_OFF : 0) | | 1003 | (irqs_disabled_flags(flags) ? TRACE_FLAG_IRQS_OFF : 0) | |
@@ -1068,6 +1179,7 @@ ftrace_trace_userstack(struct ring_buffer *buffer, unsigned long flags, int pc) | |||
1068 | return; | 1179 | return; |
1069 | entry = ring_buffer_event_data(event); | 1180 | entry = ring_buffer_event_data(event); |
1070 | 1181 | ||
1182 | entry->tgid = current->tgid; | ||
1071 | memset(&entry->caller, 0, sizeof(entry->caller)); | 1183 | memset(&entry->caller, 0, sizeof(entry->caller)); |
1072 | 1184 | ||
1073 | trace.nr_entries = 0; | 1185 | trace.nr_entries = 0; |
@@ -1094,6 +1206,7 @@ ftrace_trace_special(void *__tr, | |||
1094 | unsigned long arg1, unsigned long arg2, unsigned long arg3, | 1206 | unsigned long arg1, unsigned long arg2, unsigned long arg3, |
1095 | int pc) | 1207 | int pc) |
1096 | { | 1208 | { |
1209 | struct ftrace_event_call *call = &event_special; | ||
1097 | struct ring_buffer_event *event; | 1210 | struct ring_buffer_event *event; |
1098 | struct trace_array *tr = __tr; | 1211 | struct trace_array *tr = __tr; |
1099 | struct ring_buffer *buffer = tr->buffer; | 1212 | struct ring_buffer *buffer = tr->buffer; |
@@ -1107,7 +1220,9 @@ ftrace_trace_special(void *__tr, | |||
1107 | entry->arg1 = arg1; | 1220 | entry->arg1 = arg1; |
1108 | entry->arg2 = arg2; | 1221 | entry->arg2 = arg2; |
1109 | entry->arg3 = arg3; | 1222 | entry->arg3 = arg3; |
1110 | trace_buffer_unlock_commit(buffer, event, 0, pc); | 1223 | |
1224 | if (!filter_check_discard(call, entry, buffer, event)) | ||
1225 | trace_buffer_unlock_commit(buffer, event, 0, pc); | ||
1111 | } | 1226 | } |
1112 | 1227 | ||
1113 | void | 1228 | void |
@@ -1530,10 +1645,10 @@ static void print_lat_help_header(struct seq_file *m) | |||
1530 | seq_puts(m, "# | / _----=> need-resched \n"); | 1645 | seq_puts(m, "# | / _----=> need-resched \n"); |
1531 | seq_puts(m, "# || / _---=> hardirq/softirq \n"); | 1646 | seq_puts(m, "# || / _---=> hardirq/softirq \n"); |
1532 | seq_puts(m, "# ||| / _--=> preempt-depth \n"); | 1647 | seq_puts(m, "# ||| / _--=> preempt-depth \n"); |
1533 | seq_puts(m, "# |||| / \n"); | 1648 | seq_puts(m, "# |||| /_--=> lock-depth \n"); |
1534 | seq_puts(m, "# ||||| delay \n"); | 1649 | seq_puts(m, "# |||||/ delay \n"); |
1535 | seq_puts(m, "# cmd pid ||||| time | caller \n"); | 1650 | seq_puts(m, "# cmd pid |||||| time | caller \n"); |
1536 | seq_puts(m, "# \\ / ||||| \\ | / \n"); | 1651 | seq_puts(m, "# \\ / |||||| \\ | / \n"); |
1537 | } | 1652 | } |
1538 | 1653 | ||
1539 | static void print_func_help_header(struct seq_file *m) | 1654 | static void print_func_help_header(struct seq_file *m) |
diff --git a/kernel/trace/trace.h b/kernel/trace/trace.h index fa1dccb579d5..86bcff94791a 100644 --- a/kernel/trace/trace.h +++ b/kernel/trace/trace.h | |||
@@ -7,6 +7,7 @@ | |||
7 | #include <linux/clocksource.h> | 7 | #include <linux/clocksource.h> |
8 | #include <linux/ring_buffer.h> | 8 | #include <linux/ring_buffer.h> |
9 | #include <linux/mmiotrace.h> | 9 | #include <linux/mmiotrace.h> |
10 | #include <linux/tracepoint.h> | ||
10 | #include <linux/ftrace.h> | 11 | #include <linux/ftrace.h> |
11 | #include <trace/boot.h> | 12 | #include <trace/boot.h> |
12 | #include <linux/kmemtrace.h> | 13 | #include <linux/kmemtrace.h> |
@@ -42,157 +43,54 @@ enum trace_type { | |||
42 | __TRACE_LAST_TYPE, | 43 | __TRACE_LAST_TYPE, |
43 | }; | 44 | }; |
44 | 45 | ||
45 | /* | 46 | enum kmemtrace_type_id { |
46 | * Function trace entry - function address and parent function addres: | 47 | KMEMTRACE_TYPE_KMALLOC = 0, /* kmalloc() or kfree(). */ |
47 | */ | 48 | KMEMTRACE_TYPE_CACHE, /* kmem_cache_*(). */ |
48 | struct ftrace_entry { | 49 | KMEMTRACE_TYPE_PAGES, /* __get_free_pages() and friends. */ |
49 | struct trace_entry ent; | ||
50 | unsigned long ip; | ||
51 | unsigned long parent_ip; | ||
52 | }; | ||
53 | |||
54 | /* Function call entry */ | ||
55 | struct ftrace_graph_ent_entry { | ||
56 | struct trace_entry ent; | ||
57 | struct ftrace_graph_ent graph_ent; | ||
58 | }; | 50 | }; |
59 | 51 | ||
60 | /* Function return entry */ | ||
61 | struct ftrace_graph_ret_entry { | ||
62 | struct trace_entry ent; | ||
63 | struct ftrace_graph_ret ret; | ||
64 | }; | ||
65 | extern struct tracer boot_tracer; | 52 | extern struct tracer boot_tracer; |
66 | 53 | ||
67 | /* | 54 | #undef __field |
68 | * Context switch trace entry - which task (and prio) we switched from/to: | 55 | #define __field(type, item) type item; |
69 | */ | ||
70 | struct ctx_switch_entry { | ||
71 | struct trace_entry ent; | ||
72 | unsigned int prev_pid; | ||
73 | unsigned char prev_prio; | ||
74 | unsigned char prev_state; | ||
75 | unsigned int next_pid; | ||
76 | unsigned char next_prio; | ||
77 | unsigned char next_state; | ||
78 | unsigned int next_cpu; | ||
79 | }; | ||
80 | |||
81 | /* | ||
82 | * Special (free-form) trace entry: | ||
83 | */ | ||
84 | struct special_entry { | ||
85 | struct trace_entry ent; | ||
86 | unsigned long arg1; | ||
87 | unsigned long arg2; | ||
88 | unsigned long arg3; | ||
89 | }; | ||
90 | |||
91 | /* | ||
92 | * Stack-trace entry: | ||
93 | */ | ||
94 | |||
95 | #define FTRACE_STACK_ENTRIES 8 | ||
96 | |||
97 | struct stack_entry { | ||
98 | struct trace_entry ent; | ||
99 | unsigned long caller[FTRACE_STACK_ENTRIES]; | ||
100 | }; | ||
101 | |||
102 | struct userstack_entry { | ||
103 | struct trace_entry ent; | ||
104 | unsigned long caller[FTRACE_STACK_ENTRIES]; | ||
105 | }; | ||
106 | |||
107 | /* | ||
108 | * trace_printk entry: | ||
109 | */ | ||
110 | struct bprint_entry { | ||
111 | struct trace_entry ent; | ||
112 | unsigned long ip; | ||
113 | const char *fmt; | ||
114 | u32 buf[]; | ||
115 | }; | ||
116 | 56 | ||
117 | struct print_entry { | 57 | #undef __field_struct |
118 | struct trace_entry ent; | 58 | #define __field_struct(type, item) __field(type, item) |
119 | unsigned long ip; | ||
120 | char buf[]; | ||
121 | }; | ||
122 | 59 | ||
123 | #define TRACE_OLD_SIZE 88 | 60 | #undef __field_desc |
61 | #define __field_desc(type, container, item) | ||
124 | 62 | ||
125 | struct trace_field_cont { | 63 | #undef __array |
126 | unsigned char type; | 64 | #define __array(type, item, size) type item[size]; |
127 | /* Temporary till we get rid of this completely */ | ||
128 | char buf[TRACE_OLD_SIZE - 1]; | ||
129 | }; | ||
130 | 65 | ||
131 | struct trace_mmiotrace_rw { | 66 | #undef __array_desc |
132 | struct trace_entry ent; | 67 | #define __array_desc(type, container, item, size) |
133 | struct mmiotrace_rw rw; | ||
134 | }; | ||
135 | 68 | ||
136 | struct trace_mmiotrace_map { | 69 | #undef __dynamic_array |
137 | struct trace_entry ent; | 70 | #define __dynamic_array(type, item) type item[]; |
138 | struct mmiotrace_map map; | ||
139 | }; | ||
140 | 71 | ||
141 | struct trace_boot_call { | 72 | #undef F_STRUCT |
142 | struct trace_entry ent; | 73 | #define F_STRUCT(args...) args |
143 | struct boot_trace_call boot_call; | ||
144 | }; | ||
145 | 74 | ||
146 | struct trace_boot_ret { | 75 | #undef FTRACE_ENTRY |
147 | struct trace_entry ent; | 76 | #define FTRACE_ENTRY(name, struct_name, id, tstruct, print) \ |
148 | struct boot_trace_ret boot_ret; | 77 | struct struct_name { \ |
149 | }; | 78 | struct trace_entry ent; \ |
150 | 79 | tstruct \ | |
151 | #define TRACE_FUNC_SIZE 30 | 80 | } |
152 | #define TRACE_FILE_SIZE 20 | ||
153 | struct trace_branch { | ||
154 | struct trace_entry ent; | ||
155 | unsigned line; | ||
156 | char func[TRACE_FUNC_SIZE+1]; | ||
157 | char file[TRACE_FILE_SIZE+1]; | ||
158 | char correct; | ||
159 | }; | ||
160 | |||
161 | struct hw_branch_entry { | ||
162 | struct trace_entry ent; | ||
163 | u64 from; | ||
164 | u64 to; | ||
165 | }; | ||
166 | |||
167 | struct trace_power { | ||
168 | struct trace_entry ent; | ||
169 | struct power_trace state_data; | ||
170 | }; | ||
171 | 81 | ||
172 | enum kmemtrace_type_id { | 82 | #undef TP_ARGS |
173 | KMEMTRACE_TYPE_KMALLOC = 0, /* kmalloc() or kfree(). */ | 83 | #define TP_ARGS(args...) args |
174 | KMEMTRACE_TYPE_CACHE, /* kmem_cache_*(). */ | ||
175 | KMEMTRACE_TYPE_PAGES, /* __get_free_pages() and friends. */ | ||
176 | }; | ||
177 | 84 | ||
178 | struct kmemtrace_alloc_entry { | 85 | #undef FTRACE_ENTRY_DUP |
179 | struct trace_entry ent; | 86 | #define FTRACE_ENTRY_DUP(name, name_struct, id, tstruct, printk) |
180 | enum kmemtrace_type_id type_id; | ||
181 | unsigned long call_site; | ||
182 | const void *ptr; | ||
183 | size_t bytes_req; | ||
184 | size_t bytes_alloc; | ||
185 | gfp_t gfp_flags; | ||
186 | int node; | ||
187 | }; | ||
188 | 87 | ||
189 | struct kmemtrace_free_entry { | 88 | #include "trace_entries.h" |
190 | struct trace_entry ent; | ||
191 | enum kmemtrace_type_id type_id; | ||
192 | unsigned long call_site; | ||
193 | const void *ptr; | ||
194 | }; | ||
195 | 89 | ||
90 | /* | ||
91 | * syscalls are special, and need special handling, this is why | ||
92 | * they are not included in trace_entries.h | ||
93 | */ | ||
196 | struct syscall_trace_enter { | 94 | struct syscall_trace_enter { |
197 | struct trace_entry ent; | 95 | struct trace_entry ent; |
198 | int nr; | 96 | int nr; |
@@ -205,13 +103,12 @@ struct syscall_trace_exit { | |||
205 | unsigned long ret; | 103 | unsigned long ret; |
206 | }; | 104 | }; |
207 | 105 | ||
208 | |||
209 | /* | 106 | /* |
210 | * trace_flag_type is an enumeration that holds different | 107 | * trace_flag_type is an enumeration that holds different |
211 | * states when a trace occurs. These are: | 108 | * states when a trace occurs. These are: |
212 | * IRQS_OFF - interrupts were disabled | 109 | * IRQS_OFF - interrupts were disabled |
213 | * IRQS_NOSUPPORT - arch does not support irqs_disabled_flags | 110 | * IRQS_NOSUPPORT - arch does not support irqs_disabled_flags |
214 | * NEED_RESCED - reschedule is requested | 111 | * NEED_RESCHED - reschedule is requested |
215 | * HARDIRQ - inside an interrupt handler | 112 | * HARDIRQ - inside an interrupt handler |
216 | * SOFTIRQ - inside a softirq handler | 113 | * SOFTIRQ - inside a softirq handler |
217 | */ | 114 | */ |
@@ -390,7 +287,6 @@ struct tracer { | |||
390 | struct tracer *next; | 287 | struct tracer *next; |
391 | int print_max; | 288 | int print_max; |
392 | struct tracer_flags *flags; | 289 | struct tracer_flags *flags; |
393 | struct tracer_stat *stats; | ||
394 | }; | 290 | }; |
395 | 291 | ||
396 | 292 | ||
@@ -469,6 +365,7 @@ void tracing_stop_sched_switch_record(void); | |||
469 | void tracing_start_sched_switch_record(void); | 365 | void tracing_start_sched_switch_record(void); |
470 | int register_tracer(struct tracer *type); | 366 | int register_tracer(struct tracer *type); |
471 | void unregister_tracer(struct tracer *type); | 367 | void unregister_tracer(struct tracer *type); |
368 | int is_tracing_stopped(void); | ||
472 | 369 | ||
473 | extern unsigned long nsecs_to_usecs(unsigned long nsecs); | 370 | extern unsigned long nsecs_to_usecs(unsigned long nsecs); |
474 | 371 | ||
@@ -509,20 +406,6 @@ static inline void __trace_stack(struct trace_array *tr, unsigned long flags, | |||
509 | 406 | ||
510 | extern cycle_t ftrace_now(int cpu); | 407 | extern cycle_t ftrace_now(int cpu); |
511 | 408 | ||
512 | #ifdef CONFIG_CONTEXT_SWITCH_TRACER | ||
513 | typedef void | ||
514 | (*tracer_switch_func_t)(void *private, | ||
515 | void *__rq, | ||
516 | struct task_struct *prev, | ||
517 | struct task_struct *next); | ||
518 | |||
519 | struct tracer_switch_ops { | ||
520 | tracer_switch_func_t func; | ||
521 | void *private; | ||
522 | struct tracer_switch_ops *next; | ||
523 | }; | ||
524 | #endif /* CONFIG_CONTEXT_SWITCH_TRACER */ | ||
525 | |||
526 | extern void trace_find_cmdline(int pid, char comm[]); | 409 | extern void trace_find_cmdline(int pid, char comm[]); |
527 | 410 | ||
528 | #ifdef CONFIG_DYNAMIC_FTRACE | 411 | #ifdef CONFIG_DYNAMIC_FTRACE |
@@ -638,6 +521,41 @@ static inline int ftrace_trace_task(struct task_struct *task) | |||
638 | #endif | 521 | #endif |
639 | 522 | ||
640 | /* | 523 | /* |
524 | * struct trace_parser - servers for reading the user input separated by spaces | ||
525 | * @cont: set if the input is not complete - no final space char was found | ||
526 | * @buffer: holds the parsed user input | ||
527 | * @idx: user input lenght | ||
528 | * @size: buffer size | ||
529 | */ | ||
530 | struct trace_parser { | ||
531 | bool cont; | ||
532 | char *buffer; | ||
533 | unsigned idx; | ||
534 | unsigned size; | ||
535 | }; | ||
536 | |||
537 | static inline bool trace_parser_loaded(struct trace_parser *parser) | ||
538 | { | ||
539 | return (parser->idx != 0); | ||
540 | } | ||
541 | |||
542 | static inline bool trace_parser_cont(struct trace_parser *parser) | ||
543 | { | ||
544 | return parser->cont; | ||
545 | } | ||
546 | |||
547 | static inline void trace_parser_clear(struct trace_parser *parser) | ||
548 | { | ||
549 | parser->cont = false; | ||
550 | parser->idx = 0; | ||
551 | } | ||
552 | |||
553 | extern int trace_parser_get_init(struct trace_parser *parser, int size); | ||
554 | extern void trace_parser_put(struct trace_parser *parser); | ||
555 | extern int trace_get_user(struct trace_parser *parser, const char __user *ubuf, | ||
556 | size_t cnt, loff_t *ppos); | ||
557 | |||
558 | /* | ||
641 | * trace_iterator_flags is an enumeration that defines bit | 559 | * trace_iterator_flags is an enumeration that defines bit |
642 | * positions into trace_flags that controls the output. | 560 | * positions into trace_flags that controls the output. |
643 | * | 561 | * |
@@ -823,58 +741,18 @@ filter_check_discard(struct ftrace_event_call *call, void *rec, | |||
823 | return 0; | 741 | return 0; |
824 | } | 742 | } |
825 | 743 | ||
826 | #define DEFINE_COMPARISON_PRED(type) \ | ||
827 | static int filter_pred_##type(struct filter_pred *pred, void *event, \ | ||
828 | int val1, int val2) \ | ||
829 | { \ | ||
830 | type *addr = (type *)(event + pred->offset); \ | ||
831 | type val = (type)pred->val; \ | ||
832 | int match = 0; \ | ||
833 | \ | ||
834 | switch (pred->op) { \ | ||
835 | case OP_LT: \ | ||
836 | match = (*addr < val); \ | ||
837 | break; \ | ||
838 | case OP_LE: \ | ||
839 | match = (*addr <= val); \ | ||
840 | break; \ | ||
841 | case OP_GT: \ | ||
842 | match = (*addr > val); \ | ||
843 | break; \ | ||
844 | case OP_GE: \ | ||
845 | match = (*addr >= val); \ | ||
846 | break; \ | ||
847 | default: \ | ||
848 | break; \ | ||
849 | } \ | ||
850 | \ | ||
851 | return match; \ | ||
852 | } | ||
853 | |||
854 | #define DEFINE_EQUALITY_PRED(size) \ | ||
855 | static int filter_pred_##size(struct filter_pred *pred, void *event, \ | ||
856 | int val1, int val2) \ | ||
857 | { \ | ||
858 | u##size *addr = (u##size *)(event + pred->offset); \ | ||
859 | u##size val = (u##size)pred->val; \ | ||
860 | int match; \ | ||
861 | \ | ||
862 | match = (val == *addr) ^ pred->not; \ | ||
863 | \ | ||
864 | return match; \ | ||
865 | } | ||
866 | |||
867 | extern struct mutex event_mutex; | 744 | extern struct mutex event_mutex; |
868 | extern struct list_head ftrace_events; | 745 | extern struct list_head ftrace_events; |
869 | 746 | ||
870 | extern const char *__start___trace_bprintk_fmt[]; | 747 | extern const char *__start___trace_bprintk_fmt[]; |
871 | extern const char *__stop___trace_bprintk_fmt[]; | 748 | extern const char *__stop___trace_bprintk_fmt[]; |
872 | 749 | ||
873 | #undef TRACE_EVENT_FORMAT | 750 | #undef FTRACE_ENTRY |
874 | #define TRACE_EVENT_FORMAT(call, proto, args, fmt, tstruct, tpfmt) \ | 751 | #define FTRACE_ENTRY(call, struct_name, id, tstruct, print) \ |
875 | extern struct ftrace_event_call event_##call; | 752 | extern struct ftrace_event_call event_##call; |
876 | #undef TRACE_EVENT_FORMAT_NOFILTER | 753 | #undef FTRACE_ENTRY_DUP |
877 | #define TRACE_EVENT_FORMAT_NOFILTER(call, proto, args, fmt, tstruct, tpfmt) | 754 | #define FTRACE_ENTRY_DUP(call, struct_name, id, tstruct, print) \ |
878 | #include "trace_event_types.h" | 755 | FTRACE_ENTRY(call, struct_name, id, PARAMS(tstruct), PARAMS(print)) |
756 | #include "trace_entries.h" | ||
879 | 757 | ||
880 | #endif /* _LINUX_KERNEL_TRACE_H */ | 758 | #endif /* _LINUX_KERNEL_TRACE_H */ |
diff --git a/kernel/trace/trace_boot.c b/kernel/trace/trace_boot.c index 19bfc75d467e..c21d5f3956ad 100644 --- a/kernel/trace/trace_boot.c +++ b/kernel/trace/trace_boot.c | |||
@@ -129,6 +129,7 @@ struct tracer boot_tracer __read_mostly = | |||
129 | 129 | ||
130 | void trace_boot_call(struct boot_trace_call *bt, initcall_t fn) | 130 | void trace_boot_call(struct boot_trace_call *bt, initcall_t fn) |
131 | { | 131 | { |
132 | struct ftrace_event_call *call = &event_boot_call; | ||
132 | struct ring_buffer_event *event; | 133 | struct ring_buffer_event *event; |
133 | struct ring_buffer *buffer; | 134 | struct ring_buffer *buffer; |
134 | struct trace_boot_call *entry; | 135 | struct trace_boot_call *entry; |
@@ -150,13 +151,15 @@ void trace_boot_call(struct boot_trace_call *bt, initcall_t fn) | |||
150 | goto out; | 151 | goto out; |
151 | entry = ring_buffer_event_data(event); | 152 | entry = ring_buffer_event_data(event); |
152 | entry->boot_call = *bt; | 153 | entry->boot_call = *bt; |
153 | trace_buffer_unlock_commit(buffer, event, 0, 0); | 154 | if (!filter_check_discard(call, entry, buffer, event)) |
155 | trace_buffer_unlock_commit(buffer, event, 0, 0); | ||
154 | out: | 156 | out: |
155 | preempt_enable(); | 157 | preempt_enable(); |
156 | } | 158 | } |
157 | 159 | ||
158 | void trace_boot_ret(struct boot_trace_ret *bt, initcall_t fn) | 160 | void trace_boot_ret(struct boot_trace_ret *bt, initcall_t fn) |
159 | { | 161 | { |
162 | struct ftrace_event_call *call = &event_boot_ret; | ||
160 | struct ring_buffer_event *event; | 163 | struct ring_buffer_event *event; |
161 | struct ring_buffer *buffer; | 164 | struct ring_buffer *buffer; |
162 | struct trace_boot_ret *entry; | 165 | struct trace_boot_ret *entry; |
@@ -175,7 +178,8 @@ void trace_boot_ret(struct boot_trace_ret *bt, initcall_t fn) | |||
175 | goto out; | 178 | goto out; |
176 | entry = ring_buffer_event_data(event); | 179 | entry = ring_buffer_event_data(event); |
177 | entry->boot_ret = *bt; | 180 | entry->boot_ret = *bt; |
178 | trace_buffer_unlock_commit(buffer, event, 0, 0); | 181 | if (!filter_check_discard(call, entry, buffer, event)) |
182 | trace_buffer_unlock_commit(buffer, event, 0, 0); | ||
179 | out: | 183 | out: |
180 | preempt_enable(); | 184 | preempt_enable(); |
181 | } | 185 | } |
diff --git a/kernel/trace/trace_clock.c b/kernel/trace/trace_clock.c index b588fd81f7f9..20c5f92e28a8 100644 --- a/kernel/trace/trace_clock.c +++ b/kernel/trace/trace_clock.c | |||
@@ -66,10 +66,14 @@ u64 notrace trace_clock(void) | |||
66 | * Used by plugins that need globally coherent timestamps. | 66 | * Used by plugins that need globally coherent timestamps. |
67 | */ | 67 | */ |
68 | 68 | ||
69 | static u64 prev_trace_clock_time; | 69 | /* keep prev_time and lock in the same cacheline. */ |
70 | 70 | static struct { | |
71 | static raw_spinlock_t trace_clock_lock ____cacheline_aligned_in_smp = | 71 | u64 prev_time; |
72 | (raw_spinlock_t)__RAW_SPIN_LOCK_UNLOCKED; | 72 | raw_spinlock_t lock; |
73 | } trace_clock_struct ____cacheline_aligned_in_smp = | ||
74 | { | ||
75 | .lock = (raw_spinlock_t)__RAW_SPIN_LOCK_UNLOCKED, | ||
76 | }; | ||
73 | 77 | ||
74 | u64 notrace trace_clock_global(void) | 78 | u64 notrace trace_clock_global(void) |
75 | { | 79 | { |
@@ -88,19 +92,19 @@ u64 notrace trace_clock_global(void) | |||
88 | if (unlikely(in_nmi())) | 92 | if (unlikely(in_nmi())) |
89 | goto out; | 93 | goto out; |
90 | 94 | ||
91 | __raw_spin_lock(&trace_clock_lock); | 95 | __raw_spin_lock(&trace_clock_struct.lock); |
92 | 96 | ||
93 | /* | 97 | /* |
94 | * TODO: if this happens often then maybe we should reset | 98 | * TODO: if this happens often then maybe we should reset |
95 | * my_scd->clock to prev_trace_clock_time+1, to make sure | 99 | * my_scd->clock to prev_time+1, to make sure |
96 | * we start ticking with the local clock from now on? | 100 | * we start ticking with the local clock from now on? |
97 | */ | 101 | */ |
98 | if ((s64)(now - prev_trace_clock_time) < 0) | 102 | if ((s64)(now - trace_clock_struct.prev_time) < 0) |
99 | now = prev_trace_clock_time + 1; | 103 | now = trace_clock_struct.prev_time + 1; |
100 | 104 | ||
101 | prev_trace_clock_time = now; | 105 | trace_clock_struct.prev_time = now; |
102 | 106 | ||
103 | __raw_spin_unlock(&trace_clock_lock); | 107 | __raw_spin_unlock(&trace_clock_struct.lock); |
104 | 108 | ||
105 | out: | 109 | out: |
106 | raw_local_irq_restore(flags); | 110 | raw_local_irq_restore(flags); |
diff --git a/kernel/trace/trace_entries.h b/kernel/trace/trace_entries.h new file mode 100644 index 000000000000..a431748ddd6e --- /dev/null +++ b/kernel/trace/trace_entries.h | |||
@@ -0,0 +1,383 @@ | |||
1 | /* | ||
2 | * This file defines the trace event structures that go into the ring | ||
3 | * buffer directly. They are created via macros so that changes for them | ||
4 | * appear in the format file. Using macros will automate this process. | ||
5 | * | ||
6 | * The macro used to create a ftrace data structure is: | ||
7 | * | ||
8 | * FTRACE_ENTRY( name, struct_name, id, structure, print ) | ||
9 | * | ||
10 | * @name: the name used the event name, as well as the name of | ||
11 | * the directory that holds the format file. | ||
12 | * | ||
13 | * @struct_name: the name of the structure that is created. | ||
14 | * | ||
15 | * @id: The event identifier that is used to detect what event | ||
16 | * this is from the ring buffer. | ||
17 | * | ||
18 | * @structure: the structure layout | ||
19 | * | ||
20 | * - __field( type, item ) | ||
21 | * This is equivalent to declaring | ||
22 | * type item; | ||
23 | * in the structure. | ||
24 | * - __array( type, item, size ) | ||
25 | * This is equivalent to declaring | ||
26 | * type item[size]; | ||
27 | * in the structure. | ||
28 | * | ||
29 | * * for structures within structures, the format of the internal | ||
30 | * structure is layed out. This allows the internal structure | ||
31 | * to be deciphered for the format file. Although these macros | ||
32 | * may become out of sync with the internal structure, they | ||
33 | * will create a compile error if it happens. Since the | ||
34 | * internel structures are just tracing helpers, this is not | ||
35 | * an issue. | ||
36 | * | ||
37 | * When an internal structure is used, it should use: | ||
38 | * | ||
39 | * __field_struct( type, item ) | ||
40 | * | ||
41 | * instead of __field. This will prevent it from being shown in | ||
42 | * the output file. The fields in the structure should use. | ||
43 | * | ||
44 | * __field_desc( type, container, item ) | ||
45 | * __array_desc( type, container, item, len ) | ||
46 | * | ||
47 | * type, item and len are the same as __field and __array, but | ||
48 | * container is added. This is the name of the item in | ||
49 | * __field_struct that this is describing. | ||
50 | * | ||
51 | * | ||
52 | * @print: the print format shown to users in the format file. | ||
53 | */ | ||
54 | |||
55 | /* | ||
56 | * Function trace entry - function address and parent function addres: | ||
57 | */ | ||
58 | FTRACE_ENTRY(function, ftrace_entry, | ||
59 | |||
60 | TRACE_FN, | ||
61 | |||
62 | F_STRUCT( | ||
63 | __field( unsigned long, ip ) | ||
64 | __field( unsigned long, parent_ip ) | ||
65 | ), | ||
66 | |||
67 | F_printk(" %lx <-- %lx", __entry->ip, __entry->parent_ip) | ||
68 | ); | ||
69 | |||
70 | /* Function call entry */ | ||
71 | FTRACE_ENTRY(funcgraph_entry, ftrace_graph_ent_entry, | ||
72 | |||
73 | TRACE_GRAPH_ENT, | ||
74 | |||
75 | F_STRUCT( | ||
76 | __field_struct( struct ftrace_graph_ent, graph_ent ) | ||
77 | __field_desc( unsigned long, graph_ent, func ) | ||
78 | __field_desc( int, graph_ent, depth ) | ||
79 | ), | ||
80 | |||
81 | F_printk("--> %lx (%d)", __entry->func, __entry->depth) | ||
82 | ); | ||
83 | |||
84 | /* Function return entry */ | ||
85 | FTRACE_ENTRY(funcgraph_exit, ftrace_graph_ret_entry, | ||
86 | |||
87 | TRACE_GRAPH_RET, | ||
88 | |||
89 | F_STRUCT( | ||
90 | __field_struct( struct ftrace_graph_ret, ret ) | ||
91 | __field_desc( unsigned long, ret, func ) | ||
92 | __field_desc( unsigned long long, ret, calltime) | ||
93 | __field_desc( unsigned long long, ret, rettime ) | ||
94 | __field_desc( unsigned long, ret, overrun ) | ||
95 | __field_desc( int, ret, depth ) | ||
96 | ), | ||
97 | |||
98 | F_printk("<-- %lx (%d) (start: %llx end: %llx) over: %d", | ||
99 | __entry->func, __entry->depth, | ||
100 | __entry->calltime, __entry->rettime, | ||
101 | __entry->depth) | ||
102 | ); | ||
103 | |||
104 | /* | ||
105 | * Context switch trace entry - which task (and prio) we switched from/to: | ||
106 | * | ||
107 | * This is used for both wakeup and context switches. We only want | ||
108 | * to create one structure, but we need two outputs for it. | ||
109 | */ | ||
110 | #define FTRACE_CTX_FIELDS \ | ||
111 | __field( unsigned int, prev_pid ) \ | ||
112 | __field( unsigned char, prev_prio ) \ | ||
113 | __field( unsigned char, prev_state ) \ | ||
114 | __field( unsigned int, next_pid ) \ | ||
115 | __field( unsigned char, next_prio ) \ | ||
116 | __field( unsigned char, next_state ) \ | ||
117 | __field( unsigned int, next_cpu ) | ||
118 | |||
119 | FTRACE_ENTRY(context_switch, ctx_switch_entry, | ||
120 | |||
121 | TRACE_CTX, | ||
122 | |||
123 | F_STRUCT( | ||
124 | FTRACE_CTX_FIELDS | ||
125 | ), | ||
126 | |||
127 | F_printk("%u:%u:%u ==> %u:%u:%u [%03u]", | ||
128 | __entry->prev_pid, __entry->prev_prio, __entry->prev_state, | ||
129 | __entry->next_pid, __entry->next_prio, __entry->next_state, | ||
130 | __entry->next_cpu | ||
131 | ) | ||
132 | ); | ||
133 | |||
134 | /* | ||
135 | * FTRACE_ENTRY_DUP only creates the format file, it will not | ||
136 | * create another structure. | ||
137 | */ | ||
138 | FTRACE_ENTRY_DUP(wakeup, ctx_switch_entry, | ||
139 | |||
140 | TRACE_WAKE, | ||
141 | |||
142 | F_STRUCT( | ||
143 | FTRACE_CTX_FIELDS | ||
144 | ), | ||
145 | |||
146 | F_printk("%u:%u:%u ==+ %u:%u:%u [%03u]", | ||
147 | __entry->prev_pid, __entry->prev_prio, __entry->prev_state, | ||
148 | __entry->next_pid, __entry->next_prio, __entry->next_state, | ||
149 | __entry->next_cpu | ||
150 | ) | ||
151 | ); | ||
152 | |||
153 | /* | ||
154 | * Special (free-form) trace entry: | ||
155 | */ | ||
156 | FTRACE_ENTRY(special, special_entry, | ||
157 | |||
158 | TRACE_SPECIAL, | ||
159 | |||
160 | F_STRUCT( | ||
161 | __field( unsigned long, arg1 ) | ||
162 | __field( unsigned long, arg2 ) | ||
163 | __field( unsigned long, arg3 ) | ||
164 | ), | ||
165 | |||
166 | F_printk("(%08lx) (%08lx) (%08lx)", | ||
167 | __entry->arg1, __entry->arg2, __entry->arg3) | ||
168 | ); | ||
169 | |||
170 | /* | ||
171 | * Stack-trace entry: | ||
172 | */ | ||
173 | |||
174 | #define FTRACE_STACK_ENTRIES 8 | ||
175 | |||
176 | FTRACE_ENTRY(kernel_stack, stack_entry, | ||
177 | |||
178 | TRACE_STACK, | ||
179 | |||
180 | F_STRUCT( | ||
181 | __array( unsigned long, caller, FTRACE_STACK_ENTRIES ) | ||
182 | ), | ||
183 | |||
184 | F_printk("\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n" | ||
185 | "\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n", | ||
186 | __entry->caller[0], __entry->caller[1], __entry->caller[2], | ||
187 | __entry->caller[3], __entry->caller[4], __entry->caller[5], | ||
188 | __entry->caller[6], __entry->caller[7]) | ||
189 | ); | ||
190 | |||
191 | FTRACE_ENTRY(user_stack, userstack_entry, | ||
192 | |||
193 | TRACE_USER_STACK, | ||
194 | |||
195 | F_STRUCT( | ||
196 | __field( unsigned int, tgid ) | ||
197 | __array( unsigned long, caller, FTRACE_STACK_ENTRIES ) | ||
198 | ), | ||
199 | |||
200 | F_printk("\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n" | ||
201 | "\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n", | ||
202 | __entry->caller[0], __entry->caller[1], __entry->caller[2], | ||
203 | __entry->caller[3], __entry->caller[4], __entry->caller[5], | ||
204 | __entry->caller[6], __entry->caller[7]) | ||
205 | ); | ||
206 | |||
207 | /* | ||
208 | * trace_printk entry: | ||
209 | */ | ||
210 | FTRACE_ENTRY(bprint, bprint_entry, | ||
211 | |||
212 | TRACE_BPRINT, | ||
213 | |||
214 | F_STRUCT( | ||
215 | __field( unsigned long, ip ) | ||
216 | __field( const char *, fmt ) | ||
217 | __dynamic_array( u32, buf ) | ||
218 | ), | ||
219 | |||
220 | F_printk("%08lx fmt:%p", | ||
221 | __entry->ip, __entry->fmt) | ||
222 | ); | ||
223 | |||
224 | FTRACE_ENTRY(print, print_entry, | ||
225 | |||
226 | TRACE_PRINT, | ||
227 | |||
228 | F_STRUCT( | ||
229 | __field( unsigned long, ip ) | ||
230 | __dynamic_array( char, buf ) | ||
231 | ), | ||
232 | |||
233 | F_printk("%08lx %s", | ||
234 | __entry->ip, __entry->buf) | ||
235 | ); | ||
236 | |||
237 | FTRACE_ENTRY(mmiotrace_rw, trace_mmiotrace_rw, | ||
238 | |||
239 | TRACE_MMIO_RW, | ||
240 | |||
241 | F_STRUCT( | ||
242 | __field_struct( struct mmiotrace_rw, rw ) | ||
243 | __field_desc( resource_size_t, rw, phys ) | ||
244 | __field_desc( unsigned long, rw, value ) | ||
245 | __field_desc( unsigned long, rw, pc ) | ||
246 | __field_desc( int, rw, map_id ) | ||
247 | __field_desc( unsigned char, rw, opcode ) | ||
248 | __field_desc( unsigned char, rw, width ) | ||
249 | ), | ||
250 | |||
251 | F_printk("%lx %lx %lx %d %x %x", | ||
252 | (unsigned long)__entry->phys, __entry->value, __entry->pc, | ||
253 | __entry->map_id, __entry->opcode, __entry->width) | ||
254 | ); | ||
255 | |||
256 | FTRACE_ENTRY(mmiotrace_map, trace_mmiotrace_map, | ||
257 | |||
258 | TRACE_MMIO_MAP, | ||
259 | |||
260 | F_STRUCT( | ||
261 | __field_struct( struct mmiotrace_map, map ) | ||
262 | __field_desc( resource_size_t, map, phys ) | ||
263 | __field_desc( unsigned long, map, virt ) | ||
264 | __field_desc( unsigned long, map, len ) | ||
265 | __field_desc( int, map, map_id ) | ||
266 | __field_desc( unsigned char, map, opcode ) | ||
267 | ), | ||
268 | |||
269 | F_printk("%lx %lx %lx %d %x", | ||
270 | (unsigned long)__entry->phys, __entry->virt, __entry->len, | ||
271 | __entry->map_id, __entry->opcode) | ||
272 | ); | ||
273 | |||
274 | FTRACE_ENTRY(boot_call, trace_boot_call, | ||
275 | |||
276 | TRACE_BOOT_CALL, | ||
277 | |||
278 | F_STRUCT( | ||
279 | __field_struct( struct boot_trace_call, boot_call ) | ||
280 | __field_desc( pid_t, boot_call, caller ) | ||
281 | __array_desc( char, boot_call, func, KSYM_SYMBOL_LEN) | ||
282 | ), | ||
283 | |||
284 | F_printk("%d %s", __entry->caller, __entry->func) | ||
285 | ); | ||
286 | |||
287 | FTRACE_ENTRY(boot_ret, trace_boot_ret, | ||
288 | |||
289 | TRACE_BOOT_RET, | ||
290 | |||
291 | F_STRUCT( | ||
292 | __field_struct( struct boot_trace_ret, boot_ret ) | ||
293 | __array_desc( char, boot_ret, func, KSYM_SYMBOL_LEN) | ||
294 | __field_desc( int, boot_ret, result ) | ||
295 | __field_desc( unsigned long, boot_ret, duration ) | ||
296 | ), | ||
297 | |||
298 | F_printk("%s %d %lx", | ||
299 | __entry->func, __entry->result, __entry->duration) | ||
300 | ); | ||
301 | |||
302 | #define TRACE_FUNC_SIZE 30 | ||
303 | #define TRACE_FILE_SIZE 20 | ||
304 | |||
305 | FTRACE_ENTRY(branch, trace_branch, | ||
306 | |||
307 | TRACE_BRANCH, | ||
308 | |||
309 | F_STRUCT( | ||
310 | __field( unsigned int, line ) | ||
311 | __array( char, func, TRACE_FUNC_SIZE+1 ) | ||
312 | __array( char, file, TRACE_FILE_SIZE+1 ) | ||
313 | __field( char, correct ) | ||
314 | ), | ||
315 | |||
316 | F_printk("%u:%s:%s (%u)", | ||
317 | __entry->line, | ||
318 | __entry->func, __entry->file, __entry->correct) | ||
319 | ); | ||
320 | |||
321 | FTRACE_ENTRY(hw_branch, hw_branch_entry, | ||
322 | |||
323 | TRACE_HW_BRANCHES, | ||
324 | |||
325 | F_STRUCT( | ||
326 | __field( u64, from ) | ||
327 | __field( u64, to ) | ||
328 | ), | ||
329 | |||
330 | F_printk("from: %llx to: %llx", __entry->from, __entry->to) | ||
331 | ); | ||
332 | |||
333 | FTRACE_ENTRY(power, trace_power, | ||
334 | |||
335 | TRACE_POWER, | ||
336 | |||
337 | F_STRUCT( | ||
338 | __field_struct( struct power_trace, state_data ) | ||
339 | __field_desc( s64, state_data, stamp ) | ||
340 | __field_desc( s64, state_data, end ) | ||
341 | __field_desc( int, state_data, type ) | ||
342 | __field_desc( int, state_data, state ) | ||
343 | ), | ||
344 | |||
345 | F_printk("%llx->%llx type:%u state:%u", | ||
346 | __entry->stamp, __entry->end, | ||
347 | __entry->type, __entry->state) | ||
348 | ); | ||
349 | |||
350 | FTRACE_ENTRY(kmem_alloc, kmemtrace_alloc_entry, | ||
351 | |||
352 | TRACE_KMEM_ALLOC, | ||
353 | |||
354 | F_STRUCT( | ||
355 | __field( enum kmemtrace_type_id, type_id ) | ||
356 | __field( unsigned long, call_site ) | ||
357 | __field( const void *, ptr ) | ||
358 | __field( size_t, bytes_req ) | ||
359 | __field( size_t, bytes_alloc ) | ||
360 | __field( gfp_t, gfp_flags ) | ||
361 | __field( int, node ) | ||
362 | ), | ||
363 | |||
364 | F_printk("type:%u call_site:%lx ptr:%p req:%zi alloc:%zi" | ||
365 | " flags:%x node:%d", | ||
366 | __entry->type_id, __entry->call_site, __entry->ptr, | ||
367 | __entry->bytes_req, __entry->bytes_alloc, | ||
368 | __entry->gfp_flags, __entry->node) | ||
369 | ); | ||
370 | |||
371 | FTRACE_ENTRY(kmem_free, kmemtrace_free_entry, | ||
372 | |||
373 | TRACE_KMEM_FREE, | ||
374 | |||
375 | F_STRUCT( | ||
376 | __field( enum kmemtrace_type_id, type_id ) | ||
377 | __field( unsigned long, call_site ) | ||
378 | __field( const void *, ptr ) | ||
379 | ), | ||
380 | |||
381 | F_printk("type:%u call_site:%lx ptr:%p", | ||
382 | __entry->type_id, __entry->call_site, __entry->ptr) | ||
383 | ); | ||
diff --git a/kernel/trace/trace_event_profile.c b/kernel/trace/trace_event_profile.c index 11ba5bb4ed0a..55a25c933d15 100644 --- a/kernel/trace/trace_event_profile.c +++ b/kernel/trace/trace_event_profile.c | |||
@@ -5,6 +5,7 @@ | |||
5 | * | 5 | * |
6 | */ | 6 | */ |
7 | 7 | ||
8 | #include <linux/module.h> | ||
8 | #include "trace.h" | 9 | #include "trace.h" |
9 | 10 | ||
10 | int ftrace_profile_enable(int event_id) | 11 | int ftrace_profile_enable(int event_id) |
@@ -14,7 +15,8 @@ int ftrace_profile_enable(int event_id) | |||
14 | 15 | ||
15 | mutex_lock(&event_mutex); | 16 | mutex_lock(&event_mutex); |
16 | list_for_each_entry(event, &ftrace_events, list) { | 17 | list_for_each_entry(event, &ftrace_events, list) { |
17 | if (event->id == event_id && event->profile_enable) { | 18 | if (event->id == event_id && event->profile_enable && |
19 | try_module_get(event->mod)) { | ||
18 | ret = event->profile_enable(event); | 20 | ret = event->profile_enable(event); |
19 | break; | 21 | break; |
20 | } | 22 | } |
@@ -32,6 +34,7 @@ void ftrace_profile_disable(int event_id) | |||
32 | list_for_each_entry(event, &ftrace_events, list) { | 34 | list_for_each_entry(event, &ftrace_events, list) { |
33 | if (event->id == event_id) { | 35 | if (event->id == event_id) { |
34 | event->profile_disable(event); | 36 | event->profile_disable(event); |
37 | module_put(event->mod); | ||
35 | break; | 38 | break; |
36 | } | 39 | } |
37 | } | 40 | } |
diff --git a/kernel/trace/trace_event_types.h b/kernel/trace/trace_event_types.h deleted file mode 100644 index 6db005e12487..000000000000 --- a/kernel/trace/trace_event_types.h +++ /dev/null | |||
@@ -1,178 +0,0 @@ | |||
1 | #undef TRACE_SYSTEM | ||
2 | #define TRACE_SYSTEM ftrace | ||
3 | |||
4 | /* | ||
5 | * We cheat and use the proto type field as the ID | ||
6 | * and args as the entry type (minus 'struct') | ||
7 | */ | ||
8 | TRACE_EVENT_FORMAT(function, TRACE_FN, ftrace_entry, ignore, | ||
9 | TRACE_STRUCT( | ||
10 | TRACE_FIELD(unsigned long, ip, ip) | ||
11 | TRACE_FIELD(unsigned long, parent_ip, parent_ip) | ||
12 | ), | ||
13 | TP_RAW_FMT(" %lx <-- %lx") | ||
14 | ); | ||
15 | |||
16 | TRACE_EVENT_FORMAT(funcgraph_entry, TRACE_GRAPH_ENT, | ||
17 | ftrace_graph_ent_entry, ignore, | ||
18 | TRACE_STRUCT( | ||
19 | TRACE_FIELD(unsigned long, graph_ent.func, func) | ||
20 | TRACE_FIELD(int, graph_ent.depth, depth) | ||
21 | ), | ||
22 | TP_RAW_FMT("--> %lx (%d)") | ||
23 | ); | ||
24 | |||
25 | TRACE_EVENT_FORMAT(funcgraph_exit, TRACE_GRAPH_RET, | ||
26 | ftrace_graph_ret_entry, ignore, | ||
27 | TRACE_STRUCT( | ||
28 | TRACE_FIELD(unsigned long, ret.func, func) | ||
29 | TRACE_FIELD(unsigned long long, ret.calltime, calltime) | ||
30 | TRACE_FIELD(unsigned long long, ret.rettime, rettime) | ||
31 | TRACE_FIELD(unsigned long, ret.overrun, overrun) | ||
32 | TRACE_FIELD(int, ret.depth, depth) | ||
33 | ), | ||
34 | TP_RAW_FMT("<-- %lx (%d)") | ||
35 | ); | ||
36 | |||
37 | TRACE_EVENT_FORMAT(wakeup, TRACE_WAKE, ctx_switch_entry, ignore, | ||
38 | TRACE_STRUCT( | ||
39 | TRACE_FIELD(unsigned int, prev_pid, prev_pid) | ||
40 | TRACE_FIELD(unsigned char, prev_prio, prev_prio) | ||
41 | TRACE_FIELD(unsigned char, prev_state, prev_state) | ||
42 | TRACE_FIELD(unsigned int, next_pid, next_pid) | ||
43 | TRACE_FIELD(unsigned char, next_prio, next_prio) | ||
44 | TRACE_FIELD(unsigned char, next_state, next_state) | ||
45 | TRACE_FIELD(unsigned int, next_cpu, next_cpu) | ||
46 | ), | ||
47 | TP_RAW_FMT("%u:%u:%u ==+ %u:%u:%u [%03u]") | ||
48 | ); | ||
49 | |||
50 | TRACE_EVENT_FORMAT(context_switch, TRACE_CTX, ctx_switch_entry, ignore, | ||
51 | TRACE_STRUCT( | ||
52 | TRACE_FIELD(unsigned int, prev_pid, prev_pid) | ||
53 | TRACE_FIELD(unsigned char, prev_prio, prev_prio) | ||
54 | TRACE_FIELD(unsigned char, prev_state, prev_state) | ||
55 | TRACE_FIELD(unsigned int, next_pid, next_pid) | ||
56 | TRACE_FIELD(unsigned char, next_prio, next_prio) | ||
57 | TRACE_FIELD(unsigned char, next_state, next_state) | ||
58 | TRACE_FIELD(unsigned int, next_cpu, next_cpu) | ||
59 | ), | ||
60 | TP_RAW_FMT("%u:%u:%u ==+ %u:%u:%u [%03u]") | ||
61 | ); | ||
62 | |||
63 | TRACE_EVENT_FORMAT_NOFILTER(special, TRACE_SPECIAL, special_entry, ignore, | ||
64 | TRACE_STRUCT( | ||
65 | TRACE_FIELD(unsigned long, arg1, arg1) | ||
66 | TRACE_FIELD(unsigned long, arg2, arg2) | ||
67 | TRACE_FIELD(unsigned long, arg3, arg3) | ||
68 | ), | ||
69 | TP_RAW_FMT("(%08lx) (%08lx) (%08lx)") | ||
70 | ); | ||
71 | |||
72 | /* | ||
73 | * Stack-trace entry: | ||
74 | */ | ||
75 | |||
76 | /* #define FTRACE_STACK_ENTRIES 8 */ | ||
77 | |||
78 | TRACE_EVENT_FORMAT(kernel_stack, TRACE_STACK, stack_entry, ignore, | ||
79 | TRACE_STRUCT( | ||
80 | TRACE_FIELD(unsigned long, caller[0], stack0) | ||
81 | TRACE_FIELD(unsigned long, caller[1], stack1) | ||
82 | TRACE_FIELD(unsigned long, caller[2], stack2) | ||
83 | TRACE_FIELD(unsigned long, caller[3], stack3) | ||
84 | TRACE_FIELD(unsigned long, caller[4], stack4) | ||
85 | TRACE_FIELD(unsigned long, caller[5], stack5) | ||
86 | TRACE_FIELD(unsigned long, caller[6], stack6) | ||
87 | TRACE_FIELD(unsigned long, caller[7], stack7) | ||
88 | ), | ||
89 | TP_RAW_FMT("\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n" | ||
90 | "\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n") | ||
91 | ); | ||
92 | |||
93 | TRACE_EVENT_FORMAT(user_stack, TRACE_USER_STACK, userstack_entry, ignore, | ||
94 | TRACE_STRUCT( | ||
95 | TRACE_FIELD(unsigned long, caller[0], stack0) | ||
96 | TRACE_FIELD(unsigned long, caller[1], stack1) | ||
97 | TRACE_FIELD(unsigned long, caller[2], stack2) | ||
98 | TRACE_FIELD(unsigned long, caller[3], stack3) | ||
99 | TRACE_FIELD(unsigned long, caller[4], stack4) | ||
100 | TRACE_FIELD(unsigned long, caller[5], stack5) | ||
101 | TRACE_FIELD(unsigned long, caller[6], stack6) | ||
102 | TRACE_FIELD(unsigned long, caller[7], stack7) | ||
103 | ), | ||
104 | TP_RAW_FMT("\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n" | ||
105 | "\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n\t=> (%08lx)\n") | ||
106 | ); | ||
107 | |||
108 | TRACE_EVENT_FORMAT(bprint, TRACE_BPRINT, bprint_entry, ignore, | ||
109 | TRACE_STRUCT( | ||
110 | TRACE_FIELD(unsigned long, ip, ip) | ||
111 | TRACE_FIELD(char *, fmt, fmt) | ||
112 | TRACE_FIELD_ZERO_CHAR(buf) | ||
113 | ), | ||
114 | TP_RAW_FMT("%08lx (%d) fmt:%p %s") | ||
115 | ); | ||
116 | |||
117 | TRACE_EVENT_FORMAT(print, TRACE_PRINT, print_entry, ignore, | ||
118 | TRACE_STRUCT( | ||
119 | TRACE_FIELD(unsigned long, ip, ip) | ||
120 | TRACE_FIELD_ZERO_CHAR(buf) | ||
121 | ), | ||
122 | TP_RAW_FMT("%08lx (%d) fmt:%p %s") | ||
123 | ); | ||
124 | |||
125 | TRACE_EVENT_FORMAT(branch, TRACE_BRANCH, trace_branch, ignore, | ||
126 | TRACE_STRUCT( | ||
127 | TRACE_FIELD(unsigned int, line, line) | ||
128 | TRACE_FIELD_SPECIAL(char func[TRACE_FUNC_SIZE+1], func, | ||
129 | TRACE_FUNC_SIZE+1, func) | ||
130 | TRACE_FIELD_SPECIAL(char file[TRACE_FUNC_SIZE+1], file, | ||
131 | TRACE_FUNC_SIZE+1, file) | ||
132 | TRACE_FIELD(char, correct, correct) | ||
133 | ), | ||
134 | TP_RAW_FMT("%u:%s:%s (%u)") | ||
135 | ); | ||
136 | |||
137 | TRACE_EVENT_FORMAT(hw_branch, TRACE_HW_BRANCHES, hw_branch_entry, ignore, | ||
138 | TRACE_STRUCT( | ||
139 | TRACE_FIELD(u64, from, from) | ||
140 | TRACE_FIELD(u64, to, to) | ||
141 | ), | ||
142 | TP_RAW_FMT("from: %llx to: %llx") | ||
143 | ); | ||
144 | |||
145 | TRACE_EVENT_FORMAT(power, TRACE_POWER, trace_power, ignore, | ||
146 | TRACE_STRUCT( | ||
147 | TRACE_FIELD_SIGN(ktime_t, state_data.stamp, stamp, 1) | ||
148 | TRACE_FIELD_SIGN(ktime_t, state_data.end, end, 1) | ||
149 | TRACE_FIELD(int, state_data.type, type) | ||
150 | TRACE_FIELD(int, state_data.state, state) | ||
151 | ), | ||
152 | TP_RAW_FMT("%llx->%llx type:%u state:%u") | ||
153 | ); | ||
154 | |||
155 | TRACE_EVENT_FORMAT(kmem_alloc, TRACE_KMEM_ALLOC, kmemtrace_alloc_entry, ignore, | ||
156 | TRACE_STRUCT( | ||
157 | TRACE_FIELD(enum kmemtrace_type_id, type_id, type_id) | ||
158 | TRACE_FIELD(unsigned long, call_site, call_site) | ||
159 | TRACE_FIELD(const void *, ptr, ptr) | ||
160 | TRACE_FIELD(size_t, bytes_req, bytes_req) | ||
161 | TRACE_FIELD(size_t, bytes_alloc, bytes_alloc) | ||
162 | TRACE_FIELD(gfp_t, gfp_flags, gfp_flags) | ||
163 | TRACE_FIELD(int, node, node) | ||
164 | ), | ||
165 | TP_RAW_FMT("type:%u call_site:%lx ptr:%p req:%lu alloc:%lu" | ||
166 | " flags:%x node:%d") | ||
167 | ); | ||
168 | |||
169 | TRACE_EVENT_FORMAT(kmem_free, TRACE_KMEM_FREE, kmemtrace_free_entry, ignore, | ||
170 | TRACE_STRUCT( | ||
171 | TRACE_FIELD(enum kmemtrace_type_id, type_id, type_id) | ||
172 | TRACE_FIELD(unsigned long, call_site, call_site) | ||
173 | TRACE_FIELD(const void *, ptr, ptr) | ||
174 | ), | ||
175 | TP_RAW_FMT("type:%u call_site:%lx ptr:%p") | ||
176 | ); | ||
177 | |||
178 | #undef TRACE_SYSTEM | ||
diff --git a/kernel/trace/trace_events.c b/kernel/trace/trace_events.c index 97e2c4d2e9eb..56c260b83a9c 100644 --- a/kernel/trace/trace_events.c +++ b/kernel/trace/trace_events.c | |||
@@ -21,6 +21,7 @@ | |||
21 | 21 | ||
22 | #include "trace_output.h" | 22 | #include "trace_output.h" |
23 | 23 | ||
24 | #undef TRACE_SYSTEM | ||
24 | #define TRACE_SYSTEM "TRACE_SYSTEM" | 25 | #define TRACE_SYSTEM "TRACE_SYSTEM" |
25 | 26 | ||
26 | DEFINE_MUTEX(event_mutex); | 27 | DEFINE_MUTEX(event_mutex); |
@@ -86,7 +87,7 @@ int trace_define_common_fields(struct ftrace_event_call *call) | |||
86 | __common_field(unsigned char, flags); | 87 | __common_field(unsigned char, flags); |
87 | __common_field(unsigned char, preempt_count); | 88 | __common_field(unsigned char, preempt_count); |
88 | __common_field(int, pid); | 89 | __common_field(int, pid); |
89 | __common_field(int, tgid); | 90 | __common_field(int, lock_depth); |
90 | 91 | ||
91 | return ret; | 92 | return ret; |
92 | } | 93 | } |
@@ -230,11 +231,9 @@ static ssize_t | |||
230 | ftrace_event_write(struct file *file, const char __user *ubuf, | 231 | ftrace_event_write(struct file *file, const char __user *ubuf, |
231 | size_t cnt, loff_t *ppos) | 232 | size_t cnt, loff_t *ppos) |
232 | { | 233 | { |
234 | struct trace_parser parser; | ||
233 | size_t read = 0; | 235 | size_t read = 0; |
234 | int i, set = 1; | ||
235 | ssize_t ret; | 236 | ssize_t ret; |
236 | char *buf; | ||
237 | char ch; | ||
238 | 237 | ||
239 | if (!cnt || cnt < 0) | 238 | if (!cnt || cnt < 0) |
240 | return 0; | 239 | return 0; |
@@ -243,60 +242,28 @@ ftrace_event_write(struct file *file, const char __user *ubuf, | |||
243 | if (ret < 0) | 242 | if (ret < 0) |
244 | return ret; | 243 | return ret; |
245 | 244 | ||
246 | ret = get_user(ch, ubuf++); | 245 | if (trace_parser_get_init(&parser, EVENT_BUF_SIZE + 1)) |
247 | if (ret) | ||
248 | return ret; | ||
249 | read++; | ||
250 | cnt--; | ||
251 | |||
252 | /* skip white space */ | ||
253 | while (cnt && isspace(ch)) { | ||
254 | ret = get_user(ch, ubuf++); | ||
255 | if (ret) | ||
256 | return ret; | ||
257 | read++; | ||
258 | cnt--; | ||
259 | } | ||
260 | |||
261 | /* Only white space found? */ | ||
262 | if (isspace(ch)) { | ||
263 | file->f_pos += read; | ||
264 | ret = read; | ||
265 | return ret; | ||
266 | } | ||
267 | |||
268 | buf = kmalloc(EVENT_BUF_SIZE+1, GFP_KERNEL); | ||
269 | if (!buf) | ||
270 | return -ENOMEM; | 246 | return -ENOMEM; |
271 | 247 | ||
272 | if (cnt > EVENT_BUF_SIZE) | 248 | read = trace_get_user(&parser, ubuf, cnt, ppos); |
273 | cnt = EVENT_BUF_SIZE; | 249 | |
250 | if (trace_parser_loaded((&parser))) { | ||
251 | int set = 1; | ||
274 | 252 | ||
275 | i = 0; | 253 | if (*parser.buffer == '!') |
276 | while (cnt && !isspace(ch)) { | ||
277 | if (!i && ch == '!') | ||
278 | set = 0; | 254 | set = 0; |
279 | else | ||
280 | buf[i++] = ch; | ||
281 | 255 | ||
282 | ret = get_user(ch, ubuf++); | 256 | parser.buffer[parser.idx] = 0; |
257 | |||
258 | ret = ftrace_set_clr_event(parser.buffer + !set, set); | ||
283 | if (ret) | 259 | if (ret) |
284 | goto out_free; | 260 | goto out_put; |
285 | read++; | ||
286 | cnt--; | ||
287 | } | 261 | } |
288 | buf[i] = 0; | ||
289 | |||
290 | file->f_pos += read; | ||
291 | |||
292 | ret = ftrace_set_clr_event(buf, set); | ||
293 | if (ret) | ||
294 | goto out_free; | ||
295 | 262 | ||
296 | ret = read; | 263 | ret = read; |
297 | 264 | ||
298 | out_free: | 265 | out_put: |
299 | kfree(buf); | 266 | trace_parser_put(&parser); |
300 | 267 | ||
301 | return ret; | 268 | return ret; |
302 | } | 269 | } |
@@ -578,7 +545,7 @@ static int trace_write_header(struct trace_seq *s) | |||
578 | FIELD(unsigned char, flags), | 545 | FIELD(unsigned char, flags), |
579 | FIELD(unsigned char, preempt_count), | 546 | FIELD(unsigned char, preempt_count), |
580 | FIELD(int, pid), | 547 | FIELD(int, pid), |
581 | FIELD(int, tgid)); | 548 | FIELD(int, lock_depth)); |
582 | } | 549 | } |
583 | 550 | ||
584 | static ssize_t | 551 | static ssize_t |
@@ -1187,7 +1154,7 @@ static int trace_module_notify(struct notifier_block *self, | |||
1187 | } | 1154 | } |
1188 | #endif /* CONFIG_MODULES */ | 1155 | #endif /* CONFIG_MODULES */ |
1189 | 1156 | ||
1190 | struct notifier_block trace_module_nb = { | 1157 | static struct notifier_block trace_module_nb = { |
1191 | .notifier_call = trace_module_notify, | 1158 | .notifier_call = trace_module_notify, |
1192 | .priority = 0, | 1159 | .priority = 0, |
1193 | }; | 1160 | }; |
@@ -1359,6 +1326,18 @@ static __init void event_trace_self_tests(void) | |||
1359 | if (!call->regfunc) | 1326 | if (!call->regfunc) |
1360 | continue; | 1327 | continue; |
1361 | 1328 | ||
1329 | /* | ||
1330 | * Testing syscall events here is pretty useless, but | ||
1331 | * we still do it if configured. But this is time consuming. | ||
1332 | * What we really need is a user thread to perform the | ||
1333 | * syscalls as we test. | ||
1334 | */ | ||
1335 | #ifndef CONFIG_EVENT_TRACE_TEST_SYSCALLS | ||
1336 | if (call->system && | ||
1337 | strcmp(call->system, "syscalls") == 0) | ||
1338 | continue; | ||
1339 | #endif | ||
1340 | |||
1362 | pr_info("Testing event %s: ", call->name); | 1341 | pr_info("Testing event %s: ", call->name); |
1363 | 1342 | ||
1364 | /* | 1343 | /* |
diff --git a/kernel/trace/trace_events_filter.c b/kernel/trace/trace_events_filter.c index 93660fbbf629..23245785927f 100644 --- a/kernel/trace/trace_events_filter.c +++ b/kernel/trace/trace_events_filter.c | |||
@@ -121,6 +121,47 @@ struct filter_parse_state { | |||
121 | } operand; | 121 | } operand; |
122 | }; | 122 | }; |
123 | 123 | ||
124 | #define DEFINE_COMPARISON_PRED(type) \ | ||
125 | static int filter_pred_##type(struct filter_pred *pred, void *event, \ | ||
126 | int val1, int val2) \ | ||
127 | { \ | ||
128 | type *addr = (type *)(event + pred->offset); \ | ||
129 | type val = (type)pred->val; \ | ||
130 | int match = 0; \ | ||
131 | \ | ||
132 | switch (pred->op) { \ | ||
133 | case OP_LT: \ | ||
134 | match = (*addr < val); \ | ||
135 | break; \ | ||
136 | case OP_LE: \ | ||
137 | match = (*addr <= val); \ | ||
138 | break; \ | ||
139 | case OP_GT: \ | ||
140 | match = (*addr > val); \ | ||
141 | break; \ | ||
142 | case OP_GE: \ | ||
143 | match = (*addr >= val); \ | ||
144 | break; \ | ||
145 | default: \ | ||
146 | break; \ | ||
147 | } \ | ||
148 | \ | ||
149 | return match; \ | ||
150 | } | ||
151 | |||
152 | #define DEFINE_EQUALITY_PRED(size) \ | ||
153 | static int filter_pred_##size(struct filter_pred *pred, void *event, \ | ||
154 | int val1, int val2) \ | ||
155 | { \ | ||
156 | u##size *addr = (u##size *)(event + pred->offset); \ | ||
157 | u##size val = (u##size)pred->val; \ | ||
158 | int match; \ | ||
159 | \ | ||
160 | match = (val == *addr) ^ pred->not; \ | ||
161 | \ | ||
162 | return match; \ | ||
163 | } | ||
164 | |||
124 | DEFINE_COMPARISON_PRED(s64); | 165 | DEFINE_COMPARISON_PRED(s64); |
125 | DEFINE_COMPARISON_PRED(u64); | 166 | DEFINE_COMPARISON_PRED(u64); |
126 | DEFINE_COMPARISON_PRED(s32); | 167 | DEFINE_COMPARISON_PRED(s32); |
diff --git a/kernel/trace/trace_export.c b/kernel/trace/trace_export.c index df1bf6e48bb9..9753fcc61bc5 100644 --- a/kernel/trace/trace_export.c +++ b/kernel/trace/trace_export.c | |||
@@ -15,146 +15,125 @@ | |||
15 | 15 | ||
16 | #include "trace_output.h" | 16 | #include "trace_output.h" |
17 | 17 | ||
18 | #undef TRACE_SYSTEM | ||
19 | #define TRACE_SYSTEM ftrace | ||
18 | 20 | ||
19 | #undef TRACE_STRUCT | 21 | /* not needed for this file */ |
20 | #define TRACE_STRUCT(args...) args | 22 | #undef __field_struct |
23 | #define __field_struct(type, item) | ||
21 | 24 | ||
22 | extern void __bad_type_size(void); | 25 | #undef __field |
26 | #define __field(type, item) type item; | ||
23 | 27 | ||
24 | #undef TRACE_FIELD | 28 | #undef __field_desc |
25 | #define TRACE_FIELD(type, item, assign) \ | 29 | #define __field_desc(type, container, item) type item; |
26 | if (sizeof(type) != sizeof(field.item)) \ | 30 | |
27 | __bad_type_size(); \ | 31 | #undef __array |
32 | #define __array(type, item, size) type item[size]; | ||
33 | |||
34 | #undef __array_desc | ||
35 | #define __array_desc(type, container, item, size) type item[size]; | ||
36 | |||
37 | #undef __dynamic_array | ||
38 | #define __dynamic_array(type, item) type item[]; | ||
39 | |||
40 | #undef F_STRUCT | ||
41 | #define F_STRUCT(args...) args | ||
42 | |||
43 | #undef F_printk | ||
44 | #define F_printk(fmt, args...) fmt, args | ||
45 | |||
46 | #undef FTRACE_ENTRY | ||
47 | #define FTRACE_ENTRY(name, struct_name, id, tstruct, print) \ | ||
48 | struct ____ftrace_##name { \ | ||
49 | tstruct \ | ||
50 | }; \ | ||
51 | static void __used ____ftrace_check_##name(void) \ | ||
52 | { \ | ||
53 | struct ____ftrace_##name *__entry = NULL; \ | ||
54 | \ | ||
55 | /* force cmpile-time check on F_printk() */ \ | ||
56 | printk(print); \ | ||
57 | } | ||
58 | |||
59 | #undef FTRACE_ENTRY_DUP | ||
60 | #define FTRACE_ENTRY_DUP(name, struct_name, id, tstruct, print) \ | ||
61 | FTRACE_ENTRY(name, struct_name, id, PARAMS(tstruct), PARAMS(print)) | ||
62 | |||
63 | #include "trace_entries.h" | ||
64 | |||
65 | |||
66 | #undef __field | ||
67 | #define __field(type, item) \ | ||
28 | ret = trace_seq_printf(s, "\tfield:" #type " " #item ";\t" \ | 68 | ret = trace_seq_printf(s, "\tfield:" #type " " #item ";\t" \ |
29 | "offset:%u;\tsize:%u;\n", \ | 69 | "offset:%zu;\tsize:%zu;\n", \ |
30 | (unsigned int)offsetof(typeof(field), item), \ | 70 | offsetof(typeof(field), item), \ |
31 | (unsigned int)sizeof(field.item)); \ | 71 | sizeof(field.item)); \ |
32 | if (!ret) \ | 72 | if (!ret) \ |
33 | return 0; | 73 | return 0; |
34 | 74 | ||
75 | #undef __field_desc | ||
76 | #define __field_desc(type, container, item) \ | ||
77 | ret = trace_seq_printf(s, "\tfield:" #type " " #item ";\t" \ | ||
78 | "offset:%zu;\tsize:%zu;\n", \ | ||
79 | offsetof(typeof(field), container.item), \ | ||
80 | sizeof(field.container.item)); \ | ||
81 | if (!ret) \ | ||
82 | return 0; | ||
35 | 83 | ||
36 | #undef TRACE_FIELD_SPECIAL | 84 | #undef __array |
37 | #define TRACE_FIELD_SPECIAL(type_item, item, len, cmd) \ | 85 | #define __array(type, item, len) \ |
38 | ret = trace_seq_printf(s, "\tfield special:" #type_item ";\t" \ | 86 | ret = trace_seq_printf(s, "\tfield:" #type " " #item "[" #len "];\t" \ |
39 | "offset:%u;\tsize:%u;\n", \ | 87 | "offset:%zu;\tsize:%zu;\n", \ |
40 | (unsigned int)offsetof(typeof(field), item), \ | 88 | offsetof(typeof(field), item), \ |
41 | (unsigned int)sizeof(field.item)); \ | 89 | sizeof(field.item)); \ |
42 | if (!ret) \ | 90 | if (!ret) \ |
43 | return 0; | 91 | return 0; |
44 | 92 | ||
45 | #undef TRACE_FIELD_ZERO_CHAR | 93 | #undef __array_desc |
46 | #define TRACE_FIELD_ZERO_CHAR(item) \ | 94 | #define __array_desc(type, container, item, len) \ |
47 | ret = trace_seq_printf(s, "\tfield:char " #item ";\t" \ | 95 | ret = trace_seq_printf(s, "\tfield:" #type " " #item "[" #len "];\t" \ |
48 | "offset:%u;\tsize:0;\n", \ | 96 | "offset:%zu;\tsize:%zu;\n", \ |
49 | (unsigned int)offsetof(typeof(field), item)); \ | 97 | offsetof(typeof(field), container.item), \ |
98 | sizeof(field.container.item)); \ | ||
50 | if (!ret) \ | 99 | if (!ret) \ |
51 | return 0; | 100 | return 0; |
52 | 101 | ||
53 | #undef TRACE_FIELD_SIGN | 102 | #undef __dynamic_array |
54 | #define TRACE_FIELD_SIGN(type, item, assign, is_signed) \ | 103 | #define __dynamic_array(type, item) \ |
55 | TRACE_FIELD(type, item, assign) | 104 | ret = trace_seq_printf(s, "\tfield:" #type " " #item ";\t" \ |
105 | "offset:%zu;\tsize:0;\n", \ | ||
106 | offsetof(typeof(field), item)); \ | ||
107 | if (!ret) \ | ||
108 | return 0; | ||
56 | 109 | ||
57 | #undef TP_RAW_FMT | 110 | #undef F_printk |
58 | #define TP_RAW_FMT(args...) args | 111 | #define F_printk(fmt, args...) "%s, %s\n", #fmt, __stringify(args) |
59 | 112 | ||
60 | #undef TRACE_EVENT_FORMAT | 113 | #undef __entry |
61 | #define TRACE_EVENT_FORMAT(call, proto, args, fmt, tstruct, tpfmt) \ | 114 | #define __entry REC |
62 | static int \ | ||
63 | ftrace_format_##call(struct ftrace_event_call *unused, \ | ||
64 | struct trace_seq *s) \ | ||
65 | { \ | ||
66 | struct args field; \ | ||
67 | int ret; \ | ||
68 | \ | ||
69 | tstruct; \ | ||
70 | \ | ||
71 | trace_seq_printf(s, "\nprint fmt: \"%s\"\n", tpfmt); \ | ||
72 | \ | ||
73 | return ret; \ | ||
74 | } | ||
75 | 115 | ||
76 | #undef TRACE_EVENT_FORMAT_NOFILTER | 116 | #undef FTRACE_ENTRY |
77 | #define TRACE_EVENT_FORMAT_NOFILTER(call, proto, args, fmt, tstruct, \ | 117 | #define FTRACE_ENTRY(name, struct_name, id, tstruct, print) \ |
78 | tpfmt) \ | ||
79 | static int \ | 118 | static int \ |
80 | ftrace_format_##call(struct ftrace_event_call *unused, \ | 119 | ftrace_format_##name(struct ftrace_event_call *unused, \ |
81 | struct trace_seq *s) \ | 120 | struct trace_seq *s) \ |
82 | { \ | 121 | { \ |
83 | struct args field; \ | 122 | struct struct_name field __attribute__((unused)); \ |
84 | int ret; \ | 123 | int ret = 0; \ |
85 | \ | 124 | \ |
86 | tstruct; \ | 125 | tstruct; \ |
87 | \ | 126 | \ |
88 | trace_seq_printf(s, "\nprint fmt: \"%s\"\n", tpfmt); \ | 127 | trace_seq_printf(s, "\nprint fmt: " print); \ |
89 | \ | 128 | \ |
90 | return ret; \ | 129 | return ret; \ |
91 | } | 130 | } |
92 | 131 | ||
93 | #include "trace_event_types.h" | 132 | #include "trace_entries.h" |
94 | |||
95 | #undef TRACE_ZERO_CHAR | ||
96 | #define TRACE_ZERO_CHAR(arg) | ||
97 | |||
98 | #undef TRACE_FIELD | ||
99 | #define TRACE_FIELD(type, item, assign)\ | ||
100 | entry->item = assign; | ||
101 | |||
102 | #undef TRACE_FIELD | ||
103 | #define TRACE_FIELD(type, item, assign)\ | ||
104 | entry->item = assign; | ||
105 | |||
106 | #undef TRACE_FIELD_SIGN | ||
107 | #define TRACE_FIELD_SIGN(type, item, assign, is_signed) \ | ||
108 | TRACE_FIELD(type, item, assign) | ||
109 | |||
110 | #undef TP_CMD | ||
111 | #define TP_CMD(cmd...) cmd | ||
112 | |||
113 | #undef TRACE_ENTRY | ||
114 | #define TRACE_ENTRY entry | ||
115 | |||
116 | #undef TRACE_FIELD_SPECIAL | ||
117 | #define TRACE_FIELD_SPECIAL(type_item, item, len, cmd) \ | ||
118 | cmd; | ||
119 | |||
120 | #undef TRACE_EVENT_FORMAT | ||
121 | #define TRACE_EVENT_FORMAT(call, proto, args, fmt, tstruct, tpfmt) \ | ||
122 | int ftrace_define_fields_##call(struct ftrace_event_call *event_call); \ | ||
123 | static int ftrace_raw_init_event_##call(void); \ | ||
124 | \ | ||
125 | struct ftrace_event_call __used \ | ||
126 | __attribute__((__aligned__(4))) \ | ||
127 | __attribute__((section("_ftrace_events"))) event_##call = { \ | ||
128 | .name = #call, \ | ||
129 | .id = proto, \ | ||
130 | .system = __stringify(TRACE_SYSTEM), \ | ||
131 | .raw_init = ftrace_raw_init_event_##call, \ | ||
132 | .show_format = ftrace_format_##call, \ | ||
133 | .define_fields = ftrace_define_fields_##call, \ | ||
134 | }; \ | ||
135 | static int ftrace_raw_init_event_##call(void) \ | ||
136 | { \ | ||
137 | INIT_LIST_HEAD(&event_##call.fields); \ | ||
138 | return 0; \ | ||
139 | } \ | ||
140 | |||
141 | #undef TRACE_EVENT_FORMAT_NOFILTER | ||
142 | #define TRACE_EVENT_FORMAT_NOFILTER(call, proto, args, fmt, tstruct, \ | ||
143 | tpfmt) \ | ||
144 | \ | ||
145 | struct ftrace_event_call __used \ | ||
146 | __attribute__((__aligned__(4))) \ | ||
147 | __attribute__((section("_ftrace_events"))) event_##call = { \ | ||
148 | .name = #call, \ | ||
149 | .id = proto, \ | ||
150 | .system = __stringify(TRACE_SYSTEM), \ | ||
151 | .show_format = ftrace_format_##call, \ | ||
152 | }; | ||
153 | 133 | ||
154 | #include "trace_event_types.h" | ||
155 | 134 | ||
156 | #undef TRACE_FIELD | 135 | #undef __field |
157 | #define TRACE_FIELD(type, item, assign) \ | 136 | #define __field(type, item) \ |
158 | ret = trace_define_field(event_call, #type, #item, \ | 137 | ret = trace_define_field(event_call, #type, #item, \ |
159 | offsetof(typeof(field), item), \ | 138 | offsetof(typeof(field), item), \ |
160 | sizeof(field.item), \ | 139 | sizeof(field.item), \ |
@@ -162,32 +141,45 @@ __attribute__((section("_ftrace_events"))) event_##call = { \ | |||
162 | if (ret) \ | 141 | if (ret) \ |
163 | return ret; | 142 | return ret; |
164 | 143 | ||
165 | #undef TRACE_FIELD_SPECIAL | 144 | #undef __field_desc |
166 | #define TRACE_FIELD_SPECIAL(type, item, len, cmd) \ | 145 | #define __field_desc(type, container, item) \ |
146 | ret = trace_define_field(event_call, #type, #item, \ | ||
147 | offsetof(typeof(field), \ | ||
148 | container.item), \ | ||
149 | sizeof(field.container.item), \ | ||
150 | is_signed_type(type), FILTER_OTHER); \ | ||
151 | if (ret) \ | ||
152 | return ret; | ||
153 | |||
154 | #undef __array | ||
155 | #define __array(type, item, len) \ | ||
156 | BUILD_BUG_ON(len > MAX_FILTER_STR_VAL); \ | ||
167 | ret = trace_define_field(event_call, #type "[" #len "]", #item, \ | 157 | ret = trace_define_field(event_call, #type "[" #len "]", #item, \ |
168 | offsetof(typeof(field), item), \ | 158 | offsetof(typeof(field), item), \ |
169 | sizeof(field.item), 0, FILTER_OTHER); \ | 159 | sizeof(field.item), 0, FILTER_OTHER); \ |
170 | if (ret) \ | 160 | if (ret) \ |
171 | return ret; | 161 | return ret; |
172 | 162 | ||
173 | #undef TRACE_FIELD_SIGN | 163 | #undef __array_desc |
174 | #define TRACE_FIELD_SIGN(type, item, assign, is_signed) \ | 164 | #define __array_desc(type, container, item, len) \ |
175 | ret = trace_define_field(event_call, #type, #item, \ | 165 | BUILD_BUG_ON(len > MAX_FILTER_STR_VAL); \ |
176 | offsetof(typeof(field), item), \ | 166 | ret = trace_define_field(event_call, #type "[" #len "]", #item, \ |
177 | sizeof(field.item), is_signed, \ | 167 | offsetof(typeof(field), \ |
168 | container.item), \ | ||
169 | sizeof(field.container.item), 0, \ | ||
178 | FILTER_OTHER); \ | 170 | FILTER_OTHER); \ |
179 | if (ret) \ | 171 | if (ret) \ |
180 | return ret; | 172 | return ret; |
181 | 173 | ||
182 | #undef TRACE_FIELD_ZERO_CHAR | 174 | #undef __dynamic_array |
183 | #define TRACE_FIELD_ZERO_CHAR(item) | 175 | #define __dynamic_array(type, item) |
184 | 176 | ||
185 | #undef TRACE_EVENT_FORMAT | 177 | #undef FTRACE_ENTRY |
186 | #define TRACE_EVENT_FORMAT(call, proto, args, fmt, tstruct, tpfmt) \ | 178 | #define FTRACE_ENTRY(name, struct_name, id, tstruct, print) \ |
187 | int \ | 179 | int \ |
188 | ftrace_define_fields_##call(struct ftrace_event_call *event_call) \ | 180 | ftrace_define_fields_##name(struct ftrace_event_call *event_call) \ |
189 | { \ | 181 | { \ |
190 | struct args field; \ | 182 | struct struct_name field; \ |
191 | int ret; \ | 183 | int ret; \ |
192 | \ | 184 | \ |
193 | ret = trace_define_common_fields(event_call); \ | 185 | ret = trace_define_common_fields(event_call); \ |
@@ -199,8 +191,42 @@ ftrace_define_fields_##call(struct ftrace_event_call *event_call) \ | |||
199 | return ret; \ | 191 | return ret; \ |
200 | } | 192 | } |
201 | 193 | ||
202 | #undef TRACE_EVENT_FORMAT_NOFILTER | 194 | #include "trace_entries.h" |
203 | #define TRACE_EVENT_FORMAT_NOFILTER(call, proto, args, fmt, tstruct, \ | 195 | |
204 | tpfmt) | 196 | |
197 | #undef __field | ||
198 | #define __field(type, item) | ||
199 | |||
200 | #undef __field_desc | ||
201 | #define __field_desc(type, container, item) | ||
202 | |||
203 | #undef __array | ||
204 | #define __array(type, item, len) | ||
205 | |||
206 | #undef __array_desc | ||
207 | #define __array_desc(type, container, item, len) | ||
208 | |||
209 | #undef __dynamic_array | ||
210 | #define __dynamic_array(type, item) | ||
211 | |||
212 | #undef FTRACE_ENTRY | ||
213 | #define FTRACE_ENTRY(call, struct_name, type, tstruct, print) \ | ||
214 | static int ftrace_raw_init_event_##call(void); \ | ||
215 | \ | ||
216 | struct ftrace_event_call __used \ | ||
217 | __attribute__((__aligned__(4))) \ | ||
218 | __attribute__((section("_ftrace_events"))) event_##call = { \ | ||
219 | .name = #call, \ | ||
220 | .id = type, \ | ||
221 | .system = __stringify(TRACE_SYSTEM), \ | ||
222 | .raw_init = ftrace_raw_init_event_##call, \ | ||
223 | .show_format = ftrace_format_##call, \ | ||
224 | .define_fields = ftrace_define_fields_##call, \ | ||
225 | }; \ | ||
226 | static int ftrace_raw_init_event_##call(void) \ | ||
227 | { \ | ||
228 | INIT_LIST_HEAD(&event_##call.fields); \ | ||
229 | return 0; \ | ||
230 | } \ | ||
205 | 231 | ||
206 | #include "trace_event_types.h" | 232 | #include "trace_entries.h" |
diff --git a/kernel/trace/trace_functions.c b/kernel/trace/trace_functions.c index 5b01b94518fc..b3f3776b0cd6 100644 --- a/kernel/trace/trace_functions.c +++ b/kernel/trace/trace_functions.c | |||
@@ -290,7 +290,7 @@ ftrace_trace_onoff_print(struct seq_file *m, unsigned long ip, | |||
290 | { | 290 | { |
291 | long count = (long)data; | 291 | long count = (long)data; |
292 | 292 | ||
293 | seq_printf(m, "%pf:", (void *)ip); | 293 | seq_printf(m, "%ps:", (void *)ip); |
294 | 294 | ||
295 | if (ops == &traceon_probe_ops) | 295 | if (ops == &traceon_probe_ops) |
296 | seq_printf(m, "traceon"); | 296 | seq_printf(m, "traceon"); |
diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c index b3749a2c3132..45e6c01b2e4d 100644 --- a/kernel/trace/trace_functions_graph.c +++ b/kernel/trace/trace_functions_graph.c | |||
@@ -124,7 +124,7 @@ ftrace_pop_return_trace(struct ftrace_graph_ret *trace, unsigned long *ret, | |||
124 | if (unlikely(current->ret_stack[index].fp != frame_pointer)) { | 124 | if (unlikely(current->ret_stack[index].fp != frame_pointer)) { |
125 | ftrace_graph_stop(); | 125 | ftrace_graph_stop(); |
126 | WARN(1, "Bad frame pointer: expected %lx, received %lx\n" | 126 | WARN(1, "Bad frame pointer: expected %lx, received %lx\n" |
127 | " from func %pF return to %lx\n", | 127 | " from func %ps return to %lx\n", |
128 | current->ret_stack[index].fp, | 128 | current->ret_stack[index].fp, |
129 | frame_pointer, | 129 | frame_pointer, |
130 | (void *)current->ret_stack[index].func, | 130 | (void *)current->ret_stack[index].func, |
@@ -364,6 +364,15 @@ print_graph_proc(struct trace_seq *s, pid_t pid) | |||
364 | } | 364 | } |
365 | 365 | ||
366 | 366 | ||
367 | static enum print_line_t | ||
368 | print_graph_lat_fmt(struct trace_seq *s, struct trace_entry *entry) | ||
369 | { | ||
370 | if (!trace_seq_putc(s, ' ')) | ||
371 | return 0; | ||
372 | |||
373 | return trace_print_lat_fmt(s, entry); | ||
374 | } | ||
375 | |||
367 | /* If the pid changed since the last trace, output this event */ | 376 | /* If the pid changed since the last trace, output this event */ |
368 | static enum print_line_t | 377 | static enum print_line_t |
369 | verif_pid(struct trace_seq *s, pid_t pid, int cpu, struct fgraph_data *data) | 378 | verif_pid(struct trace_seq *s, pid_t pid, int cpu, struct fgraph_data *data) |
@@ -521,6 +530,7 @@ print_graph_irq(struct trace_iterator *iter, unsigned long addr, | |||
521 | if (ret == TRACE_TYPE_PARTIAL_LINE) | 530 | if (ret == TRACE_TYPE_PARTIAL_LINE) |
522 | return TRACE_TYPE_PARTIAL_LINE; | 531 | return TRACE_TYPE_PARTIAL_LINE; |
523 | } | 532 | } |
533 | |||
524 | /* Proc */ | 534 | /* Proc */ |
525 | if (tracer_flags.val & TRACE_GRAPH_PRINT_PROC) { | 535 | if (tracer_flags.val & TRACE_GRAPH_PRINT_PROC) { |
526 | ret = print_graph_proc(s, pid); | 536 | ret = print_graph_proc(s, pid); |
@@ -659,7 +669,7 @@ print_graph_entry_leaf(struct trace_iterator *iter, | |||
659 | return TRACE_TYPE_PARTIAL_LINE; | 669 | return TRACE_TYPE_PARTIAL_LINE; |
660 | } | 670 | } |
661 | 671 | ||
662 | ret = trace_seq_printf(s, "%pf();\n", (void *)call->func); | 672 | ret = trace_seq_printf(s, "%ps();\n", (void *)call->func); |
663 | if (!ret) | 673 | if (!ret) |
664 | return TRACE_TYPE_PARTIAL_LINE; | 674 | return TRACE_TYPE_PARTIAL_LINE; |
665 | 675 | ||
@@ -702,7 +712,7 @@ print_graph_entry_nested(struct trace_iterator *iter, | |||
702 | return TRACE_TYPE_PARTIAL_LINE; | 712 | return TRACE_TYPE_PARTIAL_LINE; |
703 | } | 713 | } |
704 | 714 | ||
705 | ret = trace_seq_printf(s, "%pf() {\n", (void *)call->func); | 715 | ret = trace_seq_printf(s, "%ps() {\n", (void *)call->func); |
706 | if (!ret) | 716 | if (!ret) |
707 | return TRACE_TYPE_PARTIAL_LINE; | 717 | return TRACE_TYPE_PARTIAL_LINE; |
708 | 718 | ||
@@ -758,6 +768,13 @@ print_graph_prologue(struct trace_iterator *iter, struct trace_seq *s, | |||
758 | return TRACE_TYPE_PARTIAL_LINE; | 768 | return TRACE_TYPE_PARTIAL_LINE; |
759 | } | 769 | } |
760 | 770 | ||
771 | /* Latency format */ | ||
772 | if (trace_flags & TRACE_ITER_LATENCY_FMT) { | ||
773 | ret = print_graph_lat_fmt(s, ent); | ||
774 | if (ret == TRACE_TYPE_PARTIAL_LINE) | ||
775 | return TRACE_TYPE_PARTIAL_LINE; | ||
776 | } | ||
777 | |||
761 | return 0; | 778 | return 0; |
762 | } | 779 | } |
763 | 780 | ||
@@ -952,28 +969,59 @@ print_graph_function(struct trace_iterator *iter) | |||
952 | return TRACE_TYPE_HANDLED; | 969 | return TRACE_TYPE_HANDLED; |
953 | } | 970 | } |
954 | 971 | ||
972 | static void print_lat_header(struct seq_file *s) | ||
973 | { | ||
974 | static const char spaces[] = " " /* 16 spaces */ | ||
975 | " " /* 4 spaces */ | ||
976 | " "; /* 17 spaces */ | ||
977 | int size = 0; | ||
978 | |||
979 | if (tracer_flags.val & TRACE_GRAPH_PRINT_ABS_TIME) | ||
980 | size += 16; | ||
981 | if (tracer_flags.val & TRACE_GRAPH_PRINT_CPU) | ||
982 | size += 4; | ||
983 | if (tracer_flags.val & TRACE_GRAPH_PRINT_PROC) | ||
984 | size += 17; | ||
985 | |||
986 | seq_printf(s, "#%.*s _-----=> irqs-off \n", size, spaces); | ||
987 | seq_printf(s, "#%.*s / _----=> need-resched \n", size, spaces); | ||
988 | seq_printf(s, "#%.*s| / _---=> hardirq/softirq \n", size, spaces); | ||
989 | seq_printf(s, "#%.*s|| / _--=> preempt-depth \n", size, spaces); | ||
990 | seq_printf(s, "#%.*s||| / _-=> lock-depth \n", size, spaces); | ||
991 | seq_printf(s, "#%.*s|||| / \n", size, spaces); | ||
992 | } | ||
993 | |||
955 | static void print_graph_headers(struct seq_file *s) | 994 | static void print_graph_headers(struct seq_file *s) |
956 | { | 995 | { |
996 | int lat = trace_flags & TRACE_ITER_LATENCY_FMT; | ||
997 | |||
998 | if (lat) | ||
999 | print_lat_header(s); | ||
1000 | |||
957 | /* 1st line */ | 1001 | /* 1st line */ |
958 | seq_printf(s, "# "); | 1002 | seq_printf(s, "#"); |
959 | if (tracer_flags.val & TRACE_GRAPH_PRINT_ABS_TIME) | 1003 | if (tracer_flags.val & TRACE_GRAPH_PRINT_ABS_TIME) |
960 | seq_printf(s, " TIME "); | 1004 | seq_printf(s, " TIME "); |
961 | if (tracer_flags.val & TRACE_GRAPH_PRINT_CPU) | 1005 | if (tracer_flags.val & TRACE_GRAPH_PRINT_CPU) |
962 | seq_printf(s, "CPU"); | 1006 | seq_printf(s, " CPU"); |
963 | if (tracer_flags.val & TRACE_GRAPH_PRINT_PROC) | 1007 | if (tracer_flags.val & TRACE_GRAPH_PRINT_PROC) |
964 | seq_printf(s, " TASK/PID "); | 1008 | seq_printf(s, " TASK/PID "); |
1009 | if (lat) | ||
1010 | seq_printf(s, "|||||"); | ||
965 | if (tracer_flags.val & TRACE_GRAPH_PRINT_DURATION) | 1011 | if (tracer_flags.val & TRACE_GRAPH_PRINT_DURATION) |
966 | seq_printf(s, " DURATION "); | 1012 | seq_printf(s, " DURATION "); |
967 | seq_printf(s, " FUNCTION CALLS\n"); | 1013 | seq_printf(s, " FUNCTION CALLS\n"); |
968 | 1014 | ||
969 | /* 2nd line */ | 1015 | /* 2nd line */ |
970 | seq_printf(s, "# "); | 1016 | seq_printf(s, "#"); |
971 | if (tracer_flags.val & TRACE_GRAPH_PRINT_ABS_TIME) | 1017 | if (tracer_flags.val & TRACE_GRAPH_PRINT_ABS_TIME) |
972 | seq_printf(s, " | "); | 1018 | seq_printf(s, " | "); |
973 | if (tracer_flags.val & TRACE_GRAPH_PRINT_CPU) | 1019 | if (tracer_flags.val & TRACE_GRAPH_PRINT_CPU) |
974 | seq_printf(s, "| "); | 1020 | seq_printf(s, " | "); |
975 | if (tracer_flags.val & TRACE_GRAPH_PRINT_PROC) | 1021 | if (tracer_flags.val & TRACE_GRAPH_PRINT_PROC) |
976 | seq_printf(s, " | | "); | 1022 | seq_printf(s, " | | "); |
1023 | if (lat) | ||
1024 | seq_printf(s, "|||||"); | ||
977 | if (tracer_flags.val & TRACE_GRAPH_PRINT_DURATION) | 1025 | if (tracer_flags.val & TRACE_GRAPH_PRINT_DURATION) |
978 | seq_printf(s, " | | "); | 1026 | seq_printf(s, " | | "); |
979 | seq_printf(s, " | | | |\n"); | 1027 | seq_printf(s, " | | | |\n"); |
diff --git a/kernel/trace/trace_irqsoff.c b/kernel/trace/trace_irqsoff.c index 5555b75a0d12..3aa7eaa2114c 100644 --- a/kernel/trace/trace_irqsoff.c +++ b/kernel/trace/trace_irqsoff.c | |||
@@ -129,15 +129,10 @@ check_critical_timing(struct trace_array *tr, | |||
129 | unsigned long parent_ip, | 129 | unsigned long parent_ip, |
130 | int cpu) | 130 | int cpu) |
131 | { | 131 | { |
132 | unsigned long latency, t0, t1; | ||
133 | cycle_t T0, T1, delta; | 132 | cycle_t T0, T1, delta; |
134 | unsigned long flags; | 133 | unsigned long flags; |
135 | int pc; | 134 | int pc; |
136 | 135 | ||
137 | /* | ||
138 | * usecs conversion is slow so we try to delay the conversion | ||
139 | * as long as possible: | ||
140 | */ | ||
141 | T0 = data->preempt_timestamp; | 136 | T0 = data->preempt_timestamp; |
142 | T1 = ftrace_now(cpu); | 137 | T1 = ftrace_now(cpu); |
143 | delta = T1-T0; | 138 | delta = T1-T0; |
@@ -157,18 +152,15 @@ check_critical_timing(struct trace_array *tr, | |||
157 | 152 | ||
158 | trace_function(tr, CALLER_ADDR0, parent_ip, flags, pc); | 153 | trace_function(tr, CALLER_ADDR0, parent_ip, flags, pc); |
159 | 154 | ||
160 | latency = nsecs_to_usecs(delta); | ||
161 | |||
162 | if (data->critical_sequence != max_sequence) | 155 | if (data->critical_sequence != max_sequence) |
163 | goto out_unlock; | 156 | goto out_unlock; |
164 | 157 | ||
165 | tracing_max_latency = delta; | ||
166 | t0 = nsecs_to_usecs(T0); | ||
167 | t1 = nsecs_to_usecs(T1); | ||
168 | |||
169 | data->critical_end = parent_ip; | 158 | data->critical_end = parent_ip; |
170 | 159 | ||
171 | update_max_tr_single(tr, current, cpu); | 160 | if (likely(!is_tracing_stopped())) { |
161 | tracing_max_latency = delta; | ||
162 | update_max_tr_single(tr, current, cpu); | ||
163 | } | ||
172 | 164 | ||
173 | max_sequence++; | 165 | max_sequence++; |
174 | 166 | ||
diff --git a/kernel/trace/trace_mmiotrace.c b/kernel/trace/trace_mmiotrace.c index c4c9bbda53d3..0acd834659ed 100644 --- a/kernel/trace/trace_mmiotrace.c +++ b/kernel/trace/trace_mmiotrace.c | |||
@@ -307,6 +307,7 @@ static void __trace_mmiotrace_rw(struct trace_array *tr, | |||
307 | struct trace_array_cpu *data, | 307 | struct trace_array_cpu *data, |
308 | struct mmiotrace_rw *rw) | 308 | struct mmiotrace_rw *rw) |
309 | { | 309 | { |
310 | struct ftrace_event_call *call = &event_mmiotrace_rw; | ||
310 | struct ring_buffer *buffer = tr->buffer; | 311 | struct ring_buffer *buffer = tr->buffer; |
311 | struct ring_buffer_event *event; | 312 | struct ring_buffer_event *event; |
312 | struct trace_mmiotrace_rw *entry; | 313 | struct trace_mmiotrace_rw *entry; |
@@ -320,7 +321,9 @@ static void __trace_mmiotrace_rw(struct trace_array *tr, | |||
320 | } | 321 | } |
321 | entry = ring_buffer_event_data(event); | 322 | entry = ring_buffer_event_data(event); |
322 | entry->rw = *rw; | 323 | entry->rw = *rw; |
323 | trace_buffer_unlock_commit(buffer, event, 0, pc); | 324 | |
325 | if (!filter_check_discard(call, entry, buffer, event)) | ||
326 | trace_buffer_unlock_commit(buffer, event, 0, pc); | ||
324 | } | 327 | } |
325 | 328 | ||
326 | void mmio_trace_rw(struct mmiotrace_rw *rw) | 329 | void mmio_trace_rw(struct mmiotrace_rw *rw) |
@@ -334,6 +337,7 @@ static void __trace_mmiotrace_map(struct trace_array *tr, | |||
334 | struct trace_array_cpu *data, | 337 | struct trace_array_cpu *data, |
335 | struct mmiotrace_map *map) | 338 | struct mmiotrace_map *map) |
336 | { | 339 | { |
340 | struct ftrace_event_call *call = &event_mmiotrace_map; | ||
337 | struct ring_buffer *buffer = tr->buffer; | 341 | struct ring_buffer *buffer = tr->buffer; |
338 | struct ring_buffer_event *event; | 342 | struct ring_buffer_event *event; |
339 | struct trace_mmiotrace_map *entry; | 343 | struct trace_mmiotrace_map *entry; |
@@ -347,7 +351,9 @@ static void __trace_mmiotrace_map(struct trace_array *tr, | |||
347 | } | 351 | } |
348 | entry = ring_buffer_event_data(event); | 352 | entry = ring_buffer_event_data(event); |
349 | entry->map = *map; | 353 | entry->map = *map; |
350 | trace_buffer_unlock_commit(buffer, event, 0, pc); | 354 | |
355 | if (!filter_check_discard(call, entry, buffer, event)) | ||
356 | trace_buffer_unlock_commit(buffer, event, 0, pc); | ||
351 | } | 357 | } |
352 | 358 | ||
353 | void mmio_trace_mapping(struct mmiotrace_map *map) | 359 | void mmio_trace_mapping(struct mmiotrace_map *map) |
diff --git a/kernel/trace/trace_output.c b/kernel/trace/trace_output.c index e0c2545622e8..f572f44c6e1e 100644 --- a/kernel/trace/trace_output.c +++ b/kernel/trace/trace_output.c | |||
@@ -407,7 +407,7 @@ seq_print_userip_objs(const struct userstack_entry *entry, struct trace_seq *s, | |||
407 | * since individual threads might have already quit! | 407 | * since individual threads might have already quit! |
408 | */ | 408 | */ |
409 | rcu_read_lock(); | 409 | rcu_read_lock(); |
410 | task = find_task_by_vpid(entry->ent.tgid); | 410 | task = find_task_by_vpid(entry->tgid); |
411 | if (task) | 411 | if (task) |
412 | mm = get_task_mm(task); | 412 | mm = get_task_mm(task); |
413 | rcu_read_unlock(); | 413 | rcu_read_unlock(); |
@@ -460,18 +460,23 @@ seq_print_ip_sym(struct trace_seq *s, unsigned long ip, unsigned long sym_flags) | |||
460 | return ret; | 460 | return ret; |
461 | } | 461 | } |
462 | 462 | ||
463 | static int | 463 | /** |
464 | lat_print_generic(struct trace_seq *s, struct trace_entry *entry, int cpu) | 464 | * trace_print_lat_fmt - print the irq, preempt and lockdep fields |
465 | * @s: trace seq struct to write to | ||
466 | * @entry: The trace entry field from the ring buffer | ||
467 | * | ||
468 | * Prints the generic fields of irqs off, in hard or softirq, preempt | ||
469 | * count and lock depth. | ||
470 | */ | ||
471 | int trace_print_lat_fmt(struct trace_seq *s, struct trace_entry *entry) | ||
465 | { | 472 | { |
466 | int hardirq, softirq; | 473 | int hardirq, softirq; |
467 | char comm[TASK_COMM_LEN]; | 474 | int ret; |
468 | 475 | ||
469 | trace_find_cmdline(entry->pid, comm); | ||
470 | hardirq = entry->flags & TRACE_FLAG_HARDIRQ; | 476 | hardirq = entry->flags & TRACE_FLAG_HARDIRQ; |
471 | softirq = entry->flags & TRACE_FLAG_SOFTIRQ; | 477 | softirq = entry->flags & TRACE_FLAG_SOFTIRQ; |
472 | 478 | ||
473 | if (!trace_seq_printf(s, "%8.8s-%-5d %3d%c%c%c", | 479 | if (!trace_seq_printf(s, "%c%c%c", |
474 | comm, entry->pid, cpu, | ||
475 | (entry->flags & TRACE_FLAG_IRQS_OFF) ? 'd' : | 480 | (entry->flags & TRACE_FLAG_IRQS_OFF) ? 'd' : |
476 | (entry->flags & TRACE_FLAG_IRQS_NOSUPPORT) ? | 481 | (entry->flags & TRACE_FLAG_IRQS_NOSUPPORT) ? |
477 | 'X' : '.', | 482 | 'X' : '.', |
@@ -481,9 +486,30 @@ lat_print_generic(struct trace_seq *s, struct trace_entry *entry, int cpu) | |||
481 | hardirq ? 'h' : softirq ? 's' : '.')) | 486 | hardirq ? 'h' : softirq ? 's' : '.')) |
482 | return 0; | 487 | return 0; |
483 | 488 | ||
489 | if (entry->lock_depth < 0) | ||
490 | ret = trace_seq_putc(s, '.'); | ||
491 | else | ||
492 | ret = trace_seq_printf(s, "%d", entry->lock_depth); | ||
493 | if (!ret) | ||
494 | return 0; | ||
495 | |||
484 | if (entry->preempt_count) | 496 | if (entry->preempt_count) |
485 | return trace_seq_printf(s, "%x", entry->preempt_count); | 497 | return trace_seq_printf(s, "%x", entry->preempt_count); |
486 | return trace_seq_puts(s, "."); | 498 | return trace_seq_putc(s, '.'); |
499 | } | ||
500 | |||
501 | static int | ||
502 | lat_print_generic(struct trace_seq *s, struct trace_entry *entry, int cpu) | ||
503 | { | ||
504 | char comm[TASK_COMM_LEN]; | ||
505 | |||
506 | trace_find_cmdline(entry->pid, comm); | ||
507 | |||
508 | if (!trace_seq_printf(s, "%8.8s-%-5d %3d", | ||
509 | comm, entry->pid, cpu)) | ||
510 | return 0; | ||
511 | |||
512 | return trace_print_lat_fmt(s, entry); | ||
487 | } | 513 | } |
488 | 514 | ||
489 | static unsigned long preempt_mark_thresh = 100; | 515 | static unsigned long preempt_mark_thresh = 100; |
diff --git a/kernel/trace/trace_output.h b/kernel/trace/trace_output.h index d38bec4a9c30..9d91c72ba38b 100644 --- a/kernel/trace/trace_output.h +++ b/kernel/trace/trace_output.h | |||
@@ -26,6 +26,8 @@ extern struct trace_event *ftrace_find_event(int type); | |||
26 | 26 | ||
27 | extern enum print_line_t trace_nop_print(struct trace_iterator *iter, | 27 | extern enum print_line_t trace_nop_print(struct trace_iterator *iter, |
28 | int flags); | 28 | int flags); |
29 | extern int | ||
30 | trace_print_lat_fmt(struct trace_seq *s, struct trace_entry *entry); | ||
29 | 31 | ||
30 | /* used by module unregistering */ | 32 | /* used by module unregistering */ |
31 | extern int __unregister_ftrace_event(struct trace_event *event); | 33 | extern int __unregister_ftrace_event(struct trace_event *event); |
diff --git a/kernel/trace/trace_sched_wakeup.c b/kernel/trace/trace_sched_wakeup.c index ad69f105a7c6..26185d727676 100644 --- a/kernel/trace/trace_sched_wakeup.c +++ b/kernel/trace/trace_sched_wakeup.c | |||
@@ -24,6 +24,7 @@ static int __read_mostly tracer_enabled; | |||
24 | 24 | ||
25 | static struct task_struct *wakeup_task; | 25 | static struct task_struct *wakeup_task; |
26 | static int wakeup_cpu; | 26 | static int wakeup_cpu; |
27 | static int wakeup_current_cpu; | ||
27 | static unsigned wakeup_prio = -1; | 28 | static unsigned wakeup_prio = -1; |
28 | static int wakeup_rt; | 29 | static int wakeup_rt; |
29 | 30 | ||
@@ -56,33 +57,23 @@ wakeup_tracer_call(unsigned long ip, unsigned long parent_ip) | |||
56 | resched = ftrace_preempt_disable(); | 57 | resched = ftrace_preempt_disable(); |
57 | 58 | ||
58 | cpu = raw_smp_processor_id(); | 59 | cpu = raw_smp_processor_id(); |
60 | if (cpu != wakeup_current_cpu) | ||
61 | goto out_enable; | ||
62 | |||
59 | data = tr->data[cpu]; | 63 | data = tr->data[cpu]; |
60 | disabled = atomic_inc_return(&data->disabled); | 64 | disabled = atomic_inc_return(&data->disabled); |
61 | if (unlikely(disabled != 1)) | 65 | if (unlikely(disabled != 1)) |
62 | goto out; | 66 | goto out; |
63 | 67 | ||
64 | local_irq_save(flags); | 68 | local_irq_save(flags); |
65 | __raw_spin_lock(&wakeup_lock); | ||
66 | |||
67 | if (unlikely(!wakeup_task)) | ||
68 | goto unlock; | ||
69 | |||
70 | /* | ||
71 | * The task can't disappear because it needs to | ||
72 | * wake up first, and we have the wakeup_lock. | ||
73 | */ | ||
74 | if (task_cpu(wakeup_task) != cpu) | ||
75 | goto unlock; | ||
76 | 69 | ||
77 | trace_function(tr, ip, parent_ip, flags, pc); | 70 | trace_function(tr, ip, parent_ip, flags, pc); |
78 | 71 | ||
79 | unlock: | ||
80 | __raw_spin_unlock(&wakeup_lock); | ||
81 | local_irq_restore(flags); | 72 | local_irq_restore(flags); |
82 | 73 | ||
83 | out: | 74 | out: |
84 | atomic_dec(&data->disabled); | 75 | atomic_dec(&data->disabled); |
85 | 76 | out_enable: | |
86 | ftrace_preempt_enable(resched); | 77 | ftrace_preempt_enable(resched); |
87 | } | 78 | } |
88 | 79 | ||
@@ -107,11 +98,18 @@ static int report_latency(cycle_t delta) | |||
107 | return 1; | 98 | return 1; |
108 | } | 99 | } |
109 | 100 | ||
101 | static void probe_wakeup_migrate_task(struct task_struct *task, int cpu) | ||
102 | { | ||
103 | if (task != wakeup_task) | ||
104 | return; | ||
105 | |||
106 | wakeup_current_cpu = cpu; | ||
107 | } | ||
108 | |||
110 | static void notrace | 109 | static void notrace |
111 | probe_wakeup_sched_switch(struct rq *rq, struct task_struct *prev, | 110 | probe_wakeup_sched_switch(struct rq *rq, struct task_struct *prev, |
112 | struct task_struct *next) | 111 | struct task_struct *next) |
113 | { | 112 | { |
114 | unsigned long latency = 0, t0 = 0, t1 = 0; | ||
115 | struct trace_array_cpu *data; | 113 | struct trace_array_cpu *data; |
116 | cycle_t T0, T1, delta; | 114 | cycle_t T0, T1, delta; |
117 | unsigned long flags; | 115 | unsigned long flags; |
@@ -157,10 +155,6 @@ probe_wakeup_sched_switch(struct rq *rq, struct task_struct *prev, | |||
157 | trace_function(wakeup_trace, CALLER_ADDR0, CALLER_ADDR1, flags, pc); | 155 | trace_function(wakeup_trace, CALLER_ADDR0, CALLER_ADDR1, flags, pc); |
158 | tracing_sched_switch_trace(wakeup_trace, prev, next, flags, pc); | 156 | tracing_sched_switch_trace(wakeup_trace, prev, next, flags, pc); |
159 | 157 | ||
160 | /* | ||
161 | * usecs conversion is slow so we try to delay the conversion | ||
162 | * as long as possible: | ||
163 | */ | ||
164 | T0 = data->preempt_timestamp; | 158 | T0 = data->preempt_timestamp; |
165 | T1 = ftrace_now(cpu); | 159 | T1 = ftrace_now(cpu); |
166 | delta = T1-T0; | 160 | delta = T1-T0; |
@@ -168,13 +162,10 @@ probe_wakeup_sched_switch(struct rq *rq, struct task_struct *prev, | |||
168 | if (!report_latency(delta)) | 162 | if (!report_latency(delta)) |
169 | goto out_unlock; | 163 | goto out_unlock; |
170 | 164 | ||
171 | latency = nsecs_to_usecs(delta); | 165 | if (likely(!is_tracing_stopped())) { |
172 | 166 | tracing_max_latency = delta; | |
173 | tracing_max_latency = delta; | 167 | update_max_tr(wakeup_trace, wakeup_task, wakeup_cpu); |
174 | t0 = nsecs_to_usecs(T0); | 168 | } |
175 | t1 = nsecs_to_usecs(T1); | ||
176 | |||
177 | update_max_tr(wakeup_trace, wakeup_task, wakeup_cpu); | ||
178 | 169 | ||
179 | out_unlock: | 170 | out_unlock: |
180 | __wakeup_reset(wakeup_trace); | 171 | __wakeup_reset(wakeup_trace); |
@@ -244,6 +235,7 @@ probe_wakeup(struct rq *rq, struct task_struct *p, int success) | |||
244 | __wakeup_reset(wakeup_trace); | 235 | __wakeup_reset(wakeup_trace); |
245 | 236 | ||
246 | wakeup_cpu = task_cpu(p); | 237 | wakeup_cpu = task_cpu(p); |
238 | wakeup_current_cpu = wakeup_cpu; | ||
247 | wakeup_prio = p->prio; | 239 | wakeup_prio = p->prio; |
248 | 240 | ||
249 | wakeup_task = p; | 241 | wakeup_task = p; |
@@ -293,6 +285,13 @@ static void start_wakeup_tracer(struct trace_array *tr) | |||
293 | goto fail_deprobe_wake_new; | 285 | goto fail_deprobe_wake_new; |
294 | } | 286 | } |
295 | 287 | ||
288 | ret = register_trace_sched_migrate_task(probe_wakeup_migrate_task); | ||
289 | if (ret) { | ||
290 | pr_info("wakeup trace: Couldn't activate tracepoint" | ||
291 | " probe to kernel_sched_migrate_task\n"); | ||
292 | return; | ||
293 | } | ||
294 | |||
296 | wakeup_reset(tr); | 295 | wakeup_reset(tr); |
297 | 296 | ||
298 | /* | 297 | /* |
@@ -325,6 +324,7 @@ static void stop_wakeup_tracer(struct trace_array *tr) | |||
325 | unregister_trace_sched_switch(probe_wakeup_sched_switch); | 324 | unregister_trace_sched_switch(probe_wakeup_sched_switch); |
326 | unregister_trace_sched_wakeup_new(probe_wakeup); | 325 | unregister_trace_sched_wakeup_new(probe_wakeup); |
327 | unregister_trace_sched_wakeup(probe_wakeup); | 326 | unregister_trace_sched_wakeup(probe_wakeup); |
327 | unregister_trace_sched_migrate_task(probe_wakeup_migrate_task); | ||
328 | } | 328 | } |
329 | 329 | ||
330 | static int __wakeup_tracer_init(struct trace_array *tr) | 330 | static int __wakeup_tracer_init(struct trace_array *tr) |
diff --git a/lib/vsprintf.c b/lib/vsprintf.c index cb8a112030bb..d320c1816a7b 100644 --- a/lib/vsprintf.c +++ b/lib/vsprintf.c | |||
@@ -581,7 +581,7 @@ static char *symbol_string(char *buf, char *end, void *ptr, | |||
581 | unsigned long value = (unsigned long) ptr; | 581 | unsigned long value = (unsigned long) ptr; |
582 | #ifdef CONFIG_KALLSYMS | 582 | #ifdef CONFIG_KALLSYMS |
583 | char sym[KSYM_SYMBOL_LEN]; | 583 | char sym[KSYM_SYMBOL_LEN]; |
584 | if (ext != 'f') | 584 | if (ext != 'f' && ext != 's') |
585 | sprint_symbol(sym, value); | 585 | sprint_symbol(sym, value); |
586 | else | 586 | else |
587 | kallsyms_lookup(value, NULL, NULL, NULL, sym); | 587 | kallsyms_lookup(value, NULL, NULL, NULL, sym); |
@@ -794,7 +794,8 @@ static char *ip4_addr_string(char *buf, char *end, const u8 *addr, | |||
794 | * | 794 | * |
795 | * - 'F' For symbolic function descriptor pointers with offset | 795 | * - 'F' For symbolic function descriptor pointers with offset |
796 | * - 'f' For simple symbolic function names without offset | 796 | * - 'f' For simple symbolic function names without offset |
797 | * - 'S' For symbolic direct pointers | 797 | * - 'S' For symbolic direct pointers with offset |
798 | * - 's' For symbolic direct pointers without offset | ||
798 | * - 'R' For a struct resource pointer, it prints the range of | 799 | * - 'R' For a struct resource pointer, it prints the range of |
799 | * addresses (not the name nor the flags) | 800 | * addresses (not the name nor the flags) |
800 | * - 'M' For a 6-byte MAC address, it prints the address in the | 801 | * - 'M' For a 6-byte MAC address, it prints the address in the |
@@ -822,6 +823,7 @@ static char *pointer(const char *fmt, char *buf, char *end, void *ptr, | |||
822 | case 'F': | 823 | case 'F': |
823 | case 'f': | 824 | case 'f': |
824 | ptr = dereference_function_descriptor(ptr); | 825 | ptr = dereference_function_descriptor(ptr); |
826 | case 's': | ||
825 | /* Fallthrough */ | 827 | /* Fallthrough */ |
826 | case 'S': | 828 | case 'S': |
827 | return symbol_string(buf, end, ptr, spec, *fmt); | 829 | return symbol_string(buf, end, ptr, spec, *fmt); |
@@ -1063,10 +1065,12 @@ qualifier: | |||
1063 | * @args: Arguments for the format string | 1065 | * @args: Arguments for the format string |
1064 | * | 1066 | * |
1065 | * This function follows C99 vsnprintf, but has some extensions: | 1067 | * This function follows C99 vsnprintf, but has some extensions: |
1066 | * %pS output the name of a text symbol | 1068 | * %pS output the name of a text symbol with offset |
1069 | * %ps output the name of a text symbol without offset | ||
1067 | * %pF output the name of a function pointer with its offset | 1070 | * %pF output the name of a function pointer with its offset |
1068 | * %pf output the name of a function pointer without its offset | 1071 | * %pf output the name of a function pointer without its offset |
1069 | * %pR output the address range in a struct resource | 1072 | * %pR output the address range in a struct resource |
1073 | * %n is ignored | ||
1070 | * | 1074 | * |
1071 | * The return value is the number of characters which would | 1075 | * The return value is the number of characters which would |
1072 | * be generated for the given input, excluding the trailing | 1076 | * be generated for the given input, excluding the trailing |
@@ -1522,11 +1526,7 @@ EXPORT_SYMBOL_GPL(vbin_printf); | |||
1522 | * a binary buffer that generated by vbin_printf. | 1526 | * a binary buffer that generated by vbin_printf. |
1523 | * | 1527 | * |
1524 | * The format follows C99 vsnprintf, but has some extensions: | 1528 | * The format follows C99 vsnprintf, but has some extensions: |
1525 | * %pS output the name of a text symbol | 1529 | * see vsnprintf comment for details. |
1526 | * %pF output the name of a function pointer with its offset | ||
1527 | * %pf output the name of a function pointer without its offset | ||
1528 | * %pR output the address range in a struct resource | ||
1529 | * %n is ignored | ||
1530 | * | 1530 | * |
1531 | * The return value is the number of characters which would | 1531 | * The return value is the number of characters which would |
1532 | * be generated for the given input, excluding the trailing | 1532 | * be generated for the given input, excluding the trailing |