summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorMauro Carvalho Chehab <mchehab@s-opensource.com>2016-09-23 13:40:40 -0400
committerMauro Carvalho Chehab <mchehab@s-opensource.com>2016-10-24 06:12:35 -0400
commit9f4b9ec63cff0048aae43b0119e64f10a7887836 (patch)
tree89197ca2450aed44260e9b33a6d12145591e4875
parent503c5bf9fa4622195bef0b46ebcc0ab6afeefed8 (diff)
Documentation/oops-tracing.txt: convert to ReST markup
- Add a document title; - use .. note:: markup; - use quote blocks where needed; - use monotonic fonts for config options and file names; - adjust whitespaces and blank lines; - replace _foo_ by **foo**; - while here, remove whitespaces at the end of paragraph; - add it to the user's book. Signed-off-by: Mauro Carvalho Chehab <mchehab@s-opensource.com>
-rw-r--r--Documentation/oops-tracing.txt255
1 files changed, 138 insertions, 117 deletions
diff --git a/Documentation/oops-tracing.txt b/Documentation/oops-tracing.txt
index f3ac05cc23e4..3e25ea7349ee 100644
--- a/Documentation/oops-tracing.txt
+++ b/Documentation/oops-tracing.txt
@@ -1,7 +1,13 @@
1NOTE: ksymoops is useless on 2.6. Please use the Oops in its original format 1OOPS tracing
2(from dmesg, etc). Ignore any references in this or other docs to "decoding 2============
3the Oops" or "running it through ksymoops". If you post an Oops from 2.6 that 3
4has been run through ksymoops, people will just tell you to repost it. 4.. note::
5
6 ``ksymoops`` is useless on 2.6 or upper. Please use the Oops in its original
7 format (from ``dmesg``, etc). Ignore any references in this or other docs to
8 "decoding the Oops" or "running it through ksymoops".
9 If you post an Oops from 2.6+ that has been run through ``ksymoops``,
10 people will just tell you to repost it.
5 11
6Quick Summary 12Quick Summary
7------------- 13-------------
@@ -12,7 +18,7 @@ If you are unsure send it to the person responsible for the code relevant to
12what you were doing. If it occurs repeatably try and describe how to recreate 18what you were doing. If it occurs repeatably try and describe how to recreate
13it. That's worth even more than the oops. 19it. That's worth even more than the oops.
14 20
15If you are totally stumped as to whom to send the report, send it to 21If you are totally stumped as to whom to send the report, send it to
16linux-kernel@vger.kernel.org. Thanks for your help in making Linux as 22linux-kernel@vger.kernel.org. Thanks for your help in making Linux as
17stable as humanly possible. 23stable as humanly possible.
18 24
@@ -20,24 +26,25 @@ Where is the Oops?
20---------------------- 26----------------------
21 27
22Normally the Oops text is read from the kernel buffers by klogd and 28Normally the Oops text is read from the kernel buffers by klogd and
23handed to syslogd which writes it to a syslog file, typically 29handed to ``syslogd`` which writes it to a syslog file, typically
24/var/log/messages (depends on /etc/syslog.conf). Sometimes klogd dies, 30``/var/log/messages`` (depends on ``/etc/syslog.conf``). Sometimes ``klogd``
25in which case you can run dmesg > file to read the data from the kernel 31dies, in which case you can run ``dmesg > file`` to read the data from the
26buffers and save it. Or you can cat /proc/kmsg > file, however you 32kernel buffers and save it. Or you can ``cat /proc/kmsg > file``, however you
27have to break in to stop the transfer, kmsg is a "never ending file". 33have to break in to stop the transfer, ``kmsg`` is a "never ending file".
28If the machine has crashed so badly that you cannot enter commands or 34If the machine has crashed so badly that you cannot enter commands or
29the disk is not available then you have three options :- 35the disk is not available then you have three options :
30 36
31(1) Hand copy the text from the screen and type it in after the machine 37(1) Hand copy the text from the screen and type it in after the machine
32 has restarted. Messy but it is the only option if you have not 38 has restarted. Messy but it is the only option if you have not
33 planned for a crash. Alternatively, you can take a picture of 39 planned for a crash. Alternatively, you can take a picture of
34 the screen with a digital camera - not nice, but better than 40 the screen with a digital camera - not nice, but better than
35 nothing. If the messages scroll off the top of the console, you 41 nothing. If the messages scroll off the top of the console, you
36 may find that booting with a higher resolution (eg, vga=791) 42 may find that booting with a higher resolution (eg, ``vga=791``)
37 will allow you to read more of the text. (Caveat: This needs vesafb, 43 will allow you to read more of the text. (Caveat: This needs ``vesafb``,
38 so won't help for 'early' oopses) 44 so won't help for 'early' oopses)
39 45
40(2) Boot with a serial console (see Documentation/serial-console.txt), 46(2) Boot with a serial console (see
47 :ref:`Documentation/serial-console.txt <serial_console>`),
41 run a null modem to a second machine and capture the output there 48 run a null modem to a second machine and capture the output there
42 using your favourite communication program. Minicom works well. 49 using your favourite communication program. Minicom works well.
43 50
@@ -49,117 +56,126 @@ the disk is not available then you have three options :-
49Full Information 56Full Information
50---------------- 57----------------
51 58
52NOTE: the message from Linus below applies to 2.4 kernel. I have preserved it 59.. note::
53for historical reasons, and because some of the information in it still 60
54applies. Especially, please ignore any references to ksymoops. 61 the message from Linus below applies to 2.4 kernel. I have preserved it
62 for historical reasons, and because some of the information in it still
63 applies. Especially, please ignore any references to ksymoops.
55 64
56From: Linus Torvalds <torvalds@osdl.org> 65 ::
57 66
58How to track down an Oops.. [originally a mail to linux-kernel] 67 From: Linus Torvalds <torvalds@osdl.org>
59 68
60The main trick is having 5 years of experience with those pesky oops 69 How to track down an Oops.. [originally a mail to linux-kernel]
61messages ;-)
62 70
63Actually, there are things you can do that make this easier. I have two 71 The main trick is having 5 years of experience with those pesky oops
64separate approaches: 72 messages ;-)
73
74Actually, there are things you can do that make this easier. I have two
75separate approaches::
65 76
66 gdb /usr/src/linux/vmlinux 77 gdb /usr/src/linux/vmlinux
67 gdb> disassemble <offending_function> 78 gdb> disassemble <offending_function>
68 79
69That's the easy way to find the problem, at least if the bug-report is 80That's the easy way to find the problem, at least if the bug-report is
70well made (like this one was - run through ksymoops to get the 81well made (like this one was - run through ``ksymoops`` to get the
71information of which function and the offset in the function that it 82information of which function and the offset in the function that it
72happened in). 83happened in).
73 84
74Oh, it helps if the report happens on a kernel that is compiled with the 85Oh, it helps if the report happens on a kernel that is compiled with the
75same compiler and similar setups. 86same compiler and similar setups.
76 87
77The other thing to do is disassemble the "Code:" part of the bug report: 88The other thing to do is disassemble the "Code:" part of the bug report:
78ksymoops will do this too with the correct tools, but if you don't have 89ksymoops will do this too with the correct tools, but if you don't have
79the tools you can just do a silly program: 90the tools you can just do a silly program::
80 91
81 char str[] = "\xXX\xXX\xXX..."; 92 char str[] = "\xXX\xXX\xXX...";
82 main(){} 93 main(){}
83 94
84and compile it with gcc -g and then do "disassemble str" (where the "XX" 95and compile it with ``gcc -g`` and then do ``disassemble str`` (where the ``XX``
85stuff are the values reported by the Oops - you can just cut-and-paste 96stuff are the values reported by the Oops - you can just cut-and-paste
86and do a replace of spaces to "\x" - that's what I do, as I'm too lazy 97and do a replace of spaces to ``\x`` - that's what I do, as I'm too lazy
87to write a program to automate this all). 98to write a program to automate this all).
88 99
89Alternatively, you can use the shell script in scripts/decodecode. 100Alternatively, you can use the shell script in ``scripts/decodecode``.
90Its usage is: decodecode < oops.txt 101Its usage is::
102
103 decodecode < oops.txt
91 104
92The hex bytes that follow "Code:" may (in some architectures) have a series 105The hex bytes that follow "Code:" may (in some architectures) have a series
93of bytes that precede the current instruction pointer as well as bytes at and 106of bytes that precede the current instruction pointer as well as bytes at and
94following the current instruction pointer. In some cases, one instruction 107following the current instruction pointer. In some cases, one instruction
95byte or word is surrounded by <> or (), as in "<86>" or "(f00d)". These 108byte or word is surrounded by ``<>`` or ``()``, as in ``<86>`` or ``(f00d)``.
96<> or () markings indicate the current instruction pointer. Example from 109These ``<>`` or ``()`` markings indicate the current instruction pointer.
97i386, split into multiple lines for readability: 110
111Example from i386, split into multiple lines for readability::
98 112
99Code: f9 0f 8d f9 00 00 00 8d 42 0c e8 dd 26 11 c7 a1 60 ea 2b f9 8b 50 08 a1 113 Code: f9 0f 8d f9 00 00 00 8d 42 0c e8 dd 26 11 c7 a1 60 ea 2b f9 8b 50 08 a1
10064 ea 2b f9 8d 34 82 8b 1e 85 db 74 6d 8b 15 60 ea 2b f9 <8b> 43 04 39 42 54 114 64 ea 2b f9 8d 34 82 8b 1e 85 db 74 6d 8b 15 60 ea 2b f9 <8b> 43 04 39 42 54
1017e 04 40 89 42 54 8b 43 04 3b 05 00 f6 52 c0 115 7e 04 40 89 42 54 8b 43 04 3b 05 00 f6 52 c0
102 116
103Finally, if you want to see where the code comes from, you can do 117Finally, if you want to see where the code comes from, you can do::
104 118
105 cd /usr/src/linux 119 cd /usr/src/linux
106 make fs/buffer.s # or whatever file the bug happened in 120 make fs/buffer.s # or whatever file the bug happened in
107 121
108and then you get a better idea of what happens than with the gdb 122and then you get a better idea of what happens than with the gdb
109disassembly. 123disassembly.
110 124
111Now, the trick is just then to combine all the data you have: the C 125Now, the trick is just then to combine all the data you have: the C
112sources (and general knowledge of what it _should_ do), the assembly 126sources (and general knowledge of what it **should** do), the assembly
113listing and the code disassembly (and additionally the register dump you 127listing and the code disassembly (and additionally the register dump you
114also get from the "oops" message - that can be useful to see _what_ the 128also get from the "oops" message - that can be useful to see **what** the
115corrupted pointers were, and when you have the assembler listing you can 129corrupted pointers were, and when you have the assembler listing you can
116also match the other registers to whatever C expressions they were used 130also match the other registers to whatever C expressions they were used
117for). 131for).
118 132
119Essentially, you just look at what doesn't match (in this case it was the 133Essentially, you just look at what doesn't match (in this case it was the
120"Code" disassembly that didn't match with what the compiler generated). 134"Code" disassembly that didn't match with what the compiler generated).
121Then you need to find out _why_ they don't match. Often it's simple - you 135Then you need to find out **why** they don't match. Often it's simple - you
122see that the code uses a NULL pointer and then you look at the code and 136see that the code uses a NULL pointer and then you look at the code and
123wonder how the NULL pointer got there, and if it's a valid thing to do 137wonder how the NULL pointer got there, and if it's a valid thing to do
124you just check against it.. 138you just check against it..
125 139
126Now, if somebody gets the idea that this is time-consuming and requires 140Now, if somebody gets the idea that this is time-consuming and requires
127some small amount of concentration, you're right. Which is why I will 141some small amount of concentration, you're right. Which is why I will
128mostly just ignore any panic reports that don't have the symbol table 142mostly just ignore any panic reports that don't have the symbol table
129info etc looked up: it simply gets too hard to look it up (I have some 143info etc looked up: it simply gets too hard to look it up (I have some
130programs to search for specific patterns in the kernel code segment, and 144programs to search for specific patterns in the kernel code segment, and
131sometimes I have been able to look up those kinds of panics too, but 145sometimes I have been able to look up those kinds of panics too, but
132that really requires pretty good knowledge of the kernel just to be able 146that really requires pretty good knowledge of the kernel just to be able
133to pick out the right sequences etc..) 147to pick out the right sequences etc..)
134 148
135_Sometimes_ it happens that I just see the disassembled code sequence 149**Sometimes** it happens that I just see the disassembled code sequence
136from the panic, and I know immediately where it's coming from. That's when 150from the panic, and I know immediately where it's coming from. That's when
137I get worried that I've been doing this for too long ;-) 151I get worried that I've been doing this for too long ;-)
138 152
139 Linus 153 Linus
140 154
141 155
142--------------------------------------------------------------------------- 156---------------------------------------------------------------------------
143Notes on Oops tracing with klogd: 157
158Notes on Oops tracing with ``klogd``
159------------------------------------
144 160
145In order to help Linus and the other kernel developers there has been 161In order to help Linus and the other kernel developers there has been
146substantial support incorporated into klogd for processing protection 162substantial support incorporated into ``klogd`` for processing protection
147faults. In order to have full support for address resolution at least 163faults. In order to have full support for address resolution at least
148version 1.3-pl3 of the sysklogd package should be used. 164version 1.3-pl3 of the ``sysklogd`` package should be used.
149 165
150When a protection fault occurs the klogd daemon automatically 166When a protection fault occurs the ``klogd`` daemon automatically
151translates important addresses in the kernel log messages to their 167translates important addresses in the kernel log messages to their
152symbolic equivalents. This translated kernel message is then 168symbolic equivalents. This translated kernel message is then
153forwarded through whatever reporting mechanism klogd is using. The 169forwarded through whatever reporting mechanism ``klogd`` is using. The
154protection fault message can be simply cut out of the message files 170protection fault message can be simply cut out of the message files
155and forwarded to the kernel developers. 171and forwarded to the kernel developers.
156 172
157Two types of address resolution are performed by klogd. The first is 173Two types of address resolution are performed by ``klogd``. The first is
158static translation and the second is dynamic translation. Static 174static translation and the second is dynamic translation. Static
159translation uses the System.map file in much the same manner that 175translation uses the System.map file in much the same manner that
160ksymoops does. In order to do static translation the klogd daemon 176ksymoops does. In order to do static translation the ``klogd`` daemon
161must be able to find a system map file at daemon initialization time. 177must be able to find a system map file at daemon initialization time.
162See the klogd man page for information on how klogd searches for map 178See the klogd man page for information on how ``klogd`` searches for map
163files. 179files.
164 180
165Dynamic address translation is important when kernel loadable modules 181Dynamic address translation is important when kernel loadable modules
@@ -178,101 +194,106 @@ information available if the developer of the loadable module chose to
178export symbol information from the module. 194export symbol information from the module.
179 195
180Since the kernel module environment can be dynamic there must be a 196Since the kernel module environment can be dynamic there must be a
181mechanism for notifying the klogd daemon when a change in module 197mechanism for notifying the ``klogd`` daemon when a change in module
182environment occurs. There are command line options available which 198environment occurs. There are command line options available which
183allow klogd to signal the currently executing daemon that symbol 199allow klogd to signal the currently executing daemon that symbol
184information should be refreshed. See the klogd manual page for more 200information should be refreshed. See the ``klogd`` manual page for more
185information. 201information.
186 202
187A patch is included with the sysklogd distribution which modifies the 203A patch is included with the sysklogd distribution which modifies the
188modules-2.0.0 package to automatically signal klogd whenever a module 204``modules-2.0.0`` package to automatically signal klogd whenever a module
189is loaded or unloaded. Applying this patch provides essentially 205is loaded or unloaded. Applying this patch provides essentially
190seamless support for debugging protection faults which occur with 206seamless support for debugging protection faults which occur with
191kernel loadable modules. 207kernel loadable modules.
192 208
193The following is an example of a protection fault in a loadable module 209The following is an example of a protection fault in a loadable module
194processed by klogd: 210processed by ``klogd``::
195--------------------------------------------------------------------------- 211
196Aug 29 09:51:01 blizard kernel: Unable to handle kernel paging request at virtual address f15e97cc 212 Aug 29 09:51:01 blizard kernel: Unable to handle kernel paging request at virtual address f15e97cc
197Aug 29 09:51:01 blizard kernel: current->tss.cr3 = 0062d000, %cr3 = 0062d000 213 Aug 29 09:51:01 blizard kernel: current->tss.cr3 = 0062d000, %cr3 = 0062d000
198Aug 29 09:51:01 blizard kernel: *pde = 00000000 214 Aug 29 09:51:01 blizard kernel: *pde = 00000000
199Aug 29 09:51:01 blizard kernel: Oops: 0002 215 Aug 29 09:51:01 blizard kernel: Oops: 0002
200Aug 29 09:51:01 blizard kernel: CPU: 0 216 Aug 29 09:51:01 blizard kernel: CPU: 0
201Aug 29 09:51:01 blizard kernel: EIP: 0010:[oops:_oops+16/3868] 217 Aug 29 09:51:01 blizard kernel: EIP: 0010:[oops:_oops+16/3868]
202Aug 29 09:51:01 blizard kernel: EFLAGS: 00010212 218 Aug 29 09:51:01 blizard kernel: EFLAGS: 00010212
203Aug 29 09:51:01 blizard kernel: eax: 315e97cc ebx: 003a6f80 ecx: 001be77b edx: 00237c0c 219 Aug 29 09:51:01 blizard kernel: eax: 315e97cc ebx: 003a6f80 ecx: 001be77b edx: 00237c0c
204Aug 29 09:51:01 blizard kernel: esi: 00000000 edi: bffffdb3 ebp: 00589f90 esp: 00589f8c 220 Aug 29 09:51:01 blizard kernel: esi: 00000000 edi: bffffdb3 ebp: 00589f90 esp: 00589f8c
205Aug 29 09:51:01 blizard kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018 221 Aug 29 09:51:01 blizard kernel: ds: 0018 es: 0018 fs: 002b gs: 002b ss: 0018
206Aug 29 09:51:01 blizard kernel: Process oops_test (pid: 3374, process nr: 21, stackpage=00589000) 222 Aug 29 09:51:01 blizard kernel: Process oops_test (pid: 3374, process nr: 21, stackpage=00589000)
207Aug 29 09:51:01 blizard kernel: Stack: 315e97cc 00589f98 0100b0b4 bffffed4 0012e38e 00240c64 003a6f80 00000001 223 Aug 29 09:51:01 blizard kernel: Stack: 315e97cc 00589f98 0100b0b4 bffffed4 0012e38e 00240c64 003a6f80 00000001
208Aug 29 09:51:01 blizard kernel: 00000000 00237810 bfffff00 0010a7fa 00000003 00000001 00000000 bfffff00 224 Aug 29 09:51:01 blizard kernel: 00000000 00237810 bfffff00 0010a7fa 00000003 00000001 00000000 bfffff00
209Aug 29 09:51:01 blizard kernel: bffffdb3 bffffed4 ffffffda 0000002b 0007002b 0000002b 0000002b 00000036 225 Aug 29 09:51:01 blizard kernel: bffffdb3 bffffed4 ffffffda 0000002b 0007002b 0000002b 0000002b 00000036
210Aug 29 09:51:01 blizard kernel: Call Trace: [oops:_oops_ioctl+48/80] [_sys_ioctl+254/272] [_system_call+82/128] 226 Aug 29 09:51:01 blizard kernel: Call Trace: [oops:_oops_ioctl+48/80] [_sys_ioctl+254/272] [_system_call+82/128]
211Aug 29 09:51:01 blizard kernel: Code: c7 00 05 00 00 00 eb 08 90 90 90 90 90 90 90 90 89 ec 5d c3 227 Aug 29 09:51:01 blizard kernel: Code: c7 00 05 00 00 00 eb 08 90 90 90 90 90 90 90 90 89 ec 5d c3
228
212--------------------------------------------------------------------------- 229---------------------------------------------------------------------------
213 230
214Dr. G.W. Wettstein Oncology Research Div. Computing Facility 231::
215Roger Maris Cancer Center INTERNET: greg@wind.rmcc.com 232
216820 4th St. N. 233 Dr. G.W. Wettstein Oncology Research Div. Computing Facility
217Fargo, ND 58122 234 Roger Maris Cancer Center INTERNET: greg@wind.rmcc.com
218Phone: 701-234-7556 235 820 4th St. N.
236 Fargo, ND 58122
237 Phone: 701-234-7556
219 238
220 239
221--------------------------------------------------------------------------- 240---------------------------------------------------------------------------
222Tainted kernels:
223 241
224Some oops reports contain the string 'Tainted: ' after the program 242Tainted kernels
243---------------
244
245Some oops reports contain the string **'Tainted: '** after the program
225counter. This indicates that the kernel has been tainted by some 246counter. This indicates that the kernel has been tainted by some
226mechanism. The string is followed by a series of position-sensitive 247mechanism. The string is followed by a series of position-sensitive
227characters, each representing a particular tainted value. 248characters, each representing a particular tainted value.
228 249
229 1: 'G' if all modules loaded have a GPL or compatible license, 'P' if 250 1) 'G' if all modules loaded have a GPL or compatible license, 'P' if
230 any proprietary module has been loaded. Modules without a 251 any proprietary module has been loaded. Modules without a
231 MODULE_LICENSE or with a MODULE_LICENSE that is not recognised by 252 MODULE_LICENSE or with a MODULE_LICENSE that is not recognised by
232 insmod as GPL compatible are assumed to be proprietary. 253 insmod as GPL compatible are assumed to be proprietary.
233 254
234 2: 'F' if any module was force loaded by "insmod -f", ' ' if all 255 2) ``F`` if any module was force loaded by ``insmod -f``, ``' '`` if all
235 modules were loaded normally. 256 modules were loaded normally.
236 257
237 3: 'S' if the oops occurred on an SMP kernel running on hardware that 258 3) ``S`` if the oops occurred on an SMP kernel running on hardware that
238 hasn't been certified as safe to run multiprocessor. 259 hasn't been certified as safe to run multiprocessor.
239 Currently this occurs only on various Athlons that are not 260 Currently this occurs only on various Athlons that are not
240 SMP capable. 261 SMP capable.
241 262
242 4: 'R' if a module was force unloaded by "rmmod -f", ' ' if all 263 4) ``R`` if a module was force unloaded by ``rmmod -f``, ``' '`` if all
243 modules were unloaded normally. 264 modules were unloaded normally.
244 265
245 5: 'M' if any processor has reported a Machine Check Exception, 266 5) ``M`` if any processor has reported a Machine Check Exception,
246 ' ' if no Machine Check Exceptions have occurred. 267 ``' '`` if no Machine Check Exceptions have occurred.
247 268
248 6: 'B' if a page-release function has found a bad page reference or 269 6) ``B`` if a page-release function has found a bad page reference or
249 some unexpected page flags. 270 some unexpected page flags.
250 271
251 7: 'U' if a user or user application specifically requested that the 272 7) ``U`` if a user or user application specifically requested that the
252 Tainted flag be set, ' ' otherwise. 273 Tainted flag be set, ``' '`` otherwise.
253 274
254 8: 'D' if the kernel has died recently, i.e. there was an OOPS or BUG. 275 8) ``D`` if the kernel has died recently, i.e. there was an OOPS or BUG.
255 276
256 9: 'A' if the ACPI table has been overridden. 277 9) ``A`` if the ACPI table has been overridden.
257 278
258 10: 'W' if a warning has previously been issued by the kernel. 279 10) ``W`` if a warning has previously been issued by the kernel.
259 (Though some warnings may set more specific taint flags.) 280 (Though some warnings may set more specific taint flags.)
260 281
261 11: 'C' if a staging driver has been loaded. 282 11) ``C`` if a staging driver has been loaded.
262 283
263 12: 'I' if the kernel is working around a severe bug in the platform 284 12) ``I`` if the kernel is working around a severe bug in the platform
264 firmware (BIOS or similar). 285 firmware (BIOS or similar).
265 286
266 13: 'O' if an externally-built ("out-of-tree") module has been loaded. 287 13) ``O`` if an externally-built ("out-of-tree") module has been loaded.
267 288
268 14: 'E' if an unsigned module has been loaded in a kernel supporting 289 14) ``E`` if an unsigned module has been loaded in a kernel supporting
269 module signature. 290 module signature.
270 291
271 15: 'L' if a soft lockup has previously occurred on the system. 292 15) ``L`` if a soft lockup has previously occurred on the system.
272 293
273 16: 'K' if the kernel has been live patched. 294 16) ``K`` if the kernel has been live patched.
274 295
275The primary reason for the 'Tainted: ' string is to tell kernel 296The primary reason for the **'Tainted: '** string is to tell kernel
276debuggers if this is a clean kernel or if anything unusual has 297debuggers if this is a clean kernel or if anything unusual has
277occurred. Tainting is permanent: even if an offending module is 298occurred. Tainting is permanent: even if an offending module is
278unloaded, the tainted value remains to indicate that the kernel is not 299unloaded, the tainted value remains to indicate that the kernel is not