aboutsummaryrefslogtreecommitdiffstats
path: root/Documentation
diff options
context:
space:
mode:
authorThomas Huth <thuth@linux.vnet.ibm.com>2014-10-28 10:12:23 -0400
committerMartin Schwidefsky <schwidefsky@de.ibm.com>2014-11-18 12:22:59 -0500
commita6b42afa3fc452339e157ad5245320804cf1206f (patch)
tree6ab5ba2a6703d52e328ed89de878ceb303e2a025 /Documentation
parentb19556231156ce3e58ffd677747bf3ef7890a937 (diff)
s390/docs: Remove sections that are not related to s390
Information how to use the GCC pre-processor, objdump, strace, top, etc. are generic and not specific to the S390 architecture, so we do not need this information in Debugging390.txt Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com> Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Diffstat (limited to 'Documentation')
-rw-r--r--Documentation/s390/Debugging390.txt374
1 files changed, 0 insertions, 374 deletions
diff --git a/Documentation/s390/Debugging390.txt b/Documentation/s390/Debugging390.txt
index 2120eec48a5c..08911b5c6b0e 100644
--- a/Documentation/s390/Debugging390.txt
+++ b/Documentation/s390/Debugging390.txt
@@ -26,11 +26,6 @@ The Linux for s/390 & z/Architecture Kernel Task Structure
26Register Usage & Stackframes on Linux for s/390 & z/Architecture 26Register Usage & Stackframes on Linux for s/390 & z/Architecture
27A sample program with comments 27A sample program with comments
28Compiling programs for debugging on Linux for s/390 & z/Architecture 28Compiling programs for debugging on Linux for s/390 & z/Architecture
29Figuring out gcc compile errors
30Debugging Tools
31objdump
32strace
33Performance Debugging
34Debugging under VM 29Debugging under VM
35s/390 & z/Architecture IO Overview 30s/390 & z/Architecture IO Overview
36Debugging IO on s/390 & z/Architecture under VM 31Debugging IO on s/390 & z/Architecture under VM
@@ -740,376 +735,7 @@ Debugging with optimisation has since much improved after fixing
740some bugs, please make sure you are using gdb-5.0 or later developed 735some bugs, please make sure you are using gdb-5.0 or later developed
741after Nov'2000. 736after Nov'2000.
742 737
743Figuring out gcc compile errors
744===============================
745If you are getting a lot of syntax errors compiling a program & the problem
746isn't blatantly obvious from the source.
747It often helps to just preprocess the file, this is done with the -E
748option in gcc.
749What this does is that it runs through the very first phase of compilation
750( compilation in gcc is done in several stages & gcc calls many programs to
751achieve its end result ) with the -E option gcc just calls the gcc preprocessor (cpp).
752The c preprocessor does the following, it joins all the files #included together
753recursively ( #include files can #include other files ) & also the c file you wish to compile.
754It puts a fully qualified path of the #included files in a comment & it
755does macro expansion.
756This is useful for debugging because
7571) You can double check whether the files you expect to be included are the ones
758that are being included ( e.g. double check that you aren't going to the i386 asm directory ).
7592) Check that macro definitions aren't clashing with typedefs,
7603) Check that definitions aren't being used before they are being included.
7614) Helps put the line emitting the error under the microscope if it contains macros.
762
763For convenience the Linux kernel's makefile will do preprocessing automatically for you
764by suffixing the file you want built with .i ( instead of .o )
765
766e.g.
767from the linux directory type
768make arch/s390/kernel/signal.i
769this will build
770
771s390-gcc -D__KERNEL__ -I/home1/barrow/linux/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer
772-fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -E arch/s390/kernel/signal.c
773> arch/s390/kernel/signal.i
774
775Now look at signal.i you should see something like.
776
777
778# 1 "/home1/barrow/linux/include/asm/types.h" 1
779typedef unsigned short umode_t;
780typedef __signed__ char __s8;
781typedef unsigned char __u8;
782typedef __signed__ short __s16;
783typedef unsigned short __u16;
784
785If instead you are getting errors further down e.g.
786unknown instruction:2515 "move.l" or better still unknown instruction:2515
787"Fixme not implemented yet, call Martin" you are probably are attempting to compile some code
788meant for another architecture or code that is simply not implemented, with a fixme statement
789stuck into the inline assembly code so that the author of the file now knows he has work to do.
790To look at the assembly emitted by gcc just before it is about to call gas ( the gnu assembler )
791use the -S option.
792Again for your convenience the Linux kernel's Makefile will hold your hand &
793do all this donkey work for you also by building the file with the .s suffix.
794e.g.
795from the Linux directory type
796make arch/s390/kernel/signal.s
797
798s390-gcc -D__KERNEL__ -I/home1/barrow/linux/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer
799-fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -S arch/s390/kernel/signal.c
800-o arch/s390/kernel/signal.s
801
802
803This will output something like, ( please note the constant pool & the useful comments
804in the prologue to give you a hand at interpreting it ).
805
806.LC54:
807 .string "misaligned (__u16 *) in __xchg\n"
808.LC57:
809 .string "misaligned (__u32 *) in __xchg\n"
810.L$PG1: # Pool sys_sigsuspend
811.LC192:
812 .long -262401
813.LC193:
814 .long -1
815.LC194:
816 .long schedule-.L$PG1
817.LC195:
818 .long do_signal-.L$PG1
819 .align 4
820.globl sys_sigsuspend
821 .type sys_sigsuspend,@function
822sys_sigsuspend:
823# leaf function 0
824# automatics 16
825# outgoing args 0
826# need frame pointer 0
827# call alloca 0
828# has varargs 0
829# incoming args (stack) 0
830# function length 168
831 STM 8,15,32(15)
832 LR 0,15
833 AHI 15,-112
834 BASR 13,0
835.L$CO1: AHI 13,.L$PG1-.L$CO1
836 ST 0,0(15)
837 LR 8,2
838 N 5,.LC192-.L$PG1(13)
839
840Adding -g to the above output makes the output even more useful
841e.g. typing
842make CC:="s390-gcc -g" kernel/sched.s
843
844which compiles.
845s390-gcc -g -D__KERNEL__ -I/home/barrow/linux-2.3/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -pipe -fno-strength-reduce -S kernel/sched.c -o kernel/sched.s
846
847also outputs stabs ( debugger ) info, from this info you can find out the
848offsets & sizes of various elements in structures.
849e.g. the stab for the structure
850struct rlimit {
851 unsigned long rlim_cur;
852 unsigned long rlim_max;
853};
854is
855.stabs "rlimit:T(151,2)=s8rlim_cur:(0,5),0,32;rlim_max:(0,5),32,32;;",128,0,0,0
856from this stab you can see that
857rlimit_cur starts at bit offset 0 & is 32 bits in size
858rlimit_max starts at bit offset 32 & is 32 bits in size.
859
860
861Debugging Tools:
862================
863
864objdump
865=======
866This is a tool with many options the most useful being ( if compiled with -g).
867objdump --source <victim program or object file> > <victims debug listing >
868
869
870The whole kernel can be compiled like this ( Doing this will make a 17MB kernel
871& a 200 MB listing ) however you have to strip it before building the image
872using the strip command to make it a more reasonable size to boot it.
873
874A source/assembly mixed dump of the kernel can be done with the line
875objdump --source vmlinux > vmlinux.lst
876Also, if the file isn't compiled -g, this will output as much debugging information
877as it can (e.g. function names). This is very slow as it spends lots
878of time searching for debugging info. The following self explanatory line should be used
879instead if the code isn't compiled -g, as it is much faster:
880objdump --disassemble-all --syms vmlinux > vmlinux.lst
881
882As hard drive space is valuable most of us use the following approach.
8831) Look at the emitted psw on the console to find the crash address in the kernel.
8842) Look at the file System.map ( in the linux directory ) produced when building
885the kernel to find the closest address less than the current PSW to find the
886offending function.
8873) use grep or similar to search the source tree looking for the source file
888 with this function if you don't know where it is.
8894) rebuild this object file with -g on, as an example suppose the file was
890( /arch/s390/kernel/signal.o )
8915) Assuming the file with the erroneous function is signal.c Move to the base of the
892Linux source tree.
8936) rm /arch/s390/kernel/signal.o
8947) make /arch/s390/kernel/signal.o
8958) watch the gcc command line emitted
8969) type it in again or alternatively cut & paste it on the console adding the -g option.
89710) objdump --source arch/s390/kernel/signal.o > signal.lst
898This will output the source & the assembly intermixed, as the snippet below shows
899This will unfortunately output addresses which aren't the same
900as the kernel ones you should be able to get around the mental arithmetic
901by playing with the --adjust-vma parameter to objdump.
902
903
904
905
906static inline void spin_lock(spinlock_t *lp)
907{
908 a0: 18 34 lr %r3,%r4
909 a2: a7 3a 03 bc ahi %r3,956
910 __asm__ __volatile(" lhi 1,-1\n"
911 a6: a7 18 ff ff lhi %r1,-1
912 aa: 1f 00 slr %r0,%r0
913 ac: ba 01 30 00 cs %r0,%r1,0(%r3)
914 b0: a7 44 ff fd jm aa <sys_sigsuspend+0x2e>
915 saveset = current->blocked;
916 b4: d2 07 f0 68 mvc 104(8,%r15),972(%r4)
917 b8: 43 cc
918 return (set->sig[0] & mask) != 0;
919}
920
9216) If debugging under VM go down to that section in the document for more info.
922
923
924I now have a tool which takes the pain out of --adjust-vma
925& you are able to do something like
926make /arch/s390/kernel/traps.lst
927& it automatically generates the correctly relocated entries for
928the text segment in traps.lst.
929This tool is now standard in linux distro's in scripts/makelst
930
931strace:
932-------
933Q. What is it ?
934A. It is a tool for intercepting calls to the kernel & logging them
935to a file & on the screen.
936
937Q. What use is it ?
938A. You can use it to find out what files a particular program opens.
939
940
941
942Example 1
943---------
944If you wanted to know does ping work but didn't have the source
945strace ping -c 1 127.0.0.1
946& then look at the man pages for each of the syscalls below,
947( In fact this is sometimes easier than looking at some spaghetti
948source which conditionally compiles for several architectures ).
949Not everything that it throws out needs to make sense immediately.
950
951Just looking quickly you can see that it is making up a RAW socket
952for the ICMP protocol.
953Doing an alarm(10) for a 10 second timeout
954& doing a gettimeofday call before & after each read to see
955how long the replies took, & writing some text to stdout so the user
956has an idea what is going on.
957
958socket(PF_INET, SOCK_RAW, IPPROTO_ICMP) = 3
959getuid() = 0
960setuid(0) = 0
961stat("/usr/share/locale/C/libc.cat", 0xbffff134) = -1 ENOENT (No such file or directory)
962stat("/usr/share/locale/libc/C", 0xbffff134) = -1 ENOENT (No such file or directory)
963stat("/usr/local/share/locale/C/libc.cat", 0xbffff134) = -1 ENOENT (No such file or directory)
964getpid() = 353
965setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0
966setsockopt(3, SOL_SOCKET, SO_RCVBUF, [49152], 4) = 0
967fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(3, 1), ...}) = 0
968mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40008000
969ioctl(1, TCGETS, {B9600 opost isig icanon echo ...}) = 0
970write(1, "PING 127.0.0.1 (127.0.0.1): 56 d"..., 42PING 127.0.0.1 (127.0.0.1): 56 data bytes
971) = 42
972sigaction(SIGINT, {0x8049ba0, [], SA_RESTART}, {SIG_DFL}) = 0
973sigaction(SIGALRM, {0x8049600, [], SA_RESTART}, {SIG_DFL}) = 0
974gettimeofday({948904719, 138951}, NULL) = 0
975sendto(3, "\10\0D\201a\1\0\0\17#\2178\307\36"..., 64, 0, {sin_family=AF_INET,
976sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, 16) = 64
977sigaction(SIGALRM, {0x8049600, [], SA_RESTART}, {0x8049600, [], SA_RESTART}) = 0
978sigaction(SIGALRM, {0x8049ba0, [], SA_RESTART}, {0x8049600, [], SA_RESTART}) = 0
979alarm(10) = 0
980recvfrom(3, "E\0\0T\0005\0\0@\1|r\177\0\0\1\177"..., 192, 0,
981{sin_family=AF_INET, sin_port=htons(50882), sin_addr=inet_addr("127.0.0.1")}, [16]) = 84
982gettimeofday({948904719, 160224}, NULL) = 0
983recvfrom(3, "E\0\0T\0006\0\0\377\1\275p\177\0"..., 192, 0,
984{sin_family=AF_INET, sin_port=htons(50882), sin_addr=inet_addr("127.0.0.1")}, [16]) = 84
985gettimeofday({948904719, 166952}, NULL) = 0
986write(1, "64 bytes from 127.0.0.1: icmp_se"...,
9875764 bytes from 127.0.0.1: icmp_seq=0 ttl=255 time=28.0 ms
988
989Example 2
990---------
991strace passwd 2>&1 | grep open
992produces the following output
993open("/etc/ld.so.cache", O_RDONLY) = 3
994open("/opt/kde/lib/libc.so.5", O_RDONLY) = -1 ENOENT (No such file or directory)
995open("/lib/libc.so.5", O_RDONLY) = 3
996open("/dev", O_RDONLY) = 3
997open("/var/run/utmp", O_RDONLY) = 3
998open("/etc/passwd", O_RDONLY) = 3
999open("/etc/shadow", O_RDONLY) = 3
1000open("/etc/login.defs", O_RDONLY) = 4
1001open("/dev/tty", O_RDONLY) = 4
1002
1003The 2>&1 is done to redirect stderr to stdout & grep is then filtering this input
1004through the pipe for each line containing the string open.
1005
1006
1007Example 3
1008---------
1009Getting sophisticated
1010telnetd crashes & I don't know why
1011
1012Steps
1013-----
10141) Replace the following line in /etc/inetd.conf
1015telnet stream tcp nowait root /usr/sbin/in.telnetd -h
1016with
1017telnet stream tcp nowait root /blah
1018
10192) Create the file /blah with the following contents to start tracing telnetd
1020#!/bin/bash
1021/usr/bin/strace -o/t1 -f /usr/sbin/in.telnetd -h
10223) chmod 700 /blah to make it executable only to root
10234)
1024killall -HUP inetd
1025or ps aux | grep inetd
1026get inetd's process id
1027& kill -HUP inetd to restart it.
1028
1029Important options
1030-----------------
1031-o is used to tell strace to output to a file in our case t1 in the root directory
1032-f is to follow children i.e.
1033e.g in our case above telnetd will start the login process & subsequently a shell like bash.
1034You will be able to tell which is which from the process ID's listed on the left hand side
1035of the strace output.
1036-p<pid> will tell strace to attach to a running process, yup this can be done provided
1037 it isn't being traced or debugged already & you have enough privileges,
1038the reason 2 processes cannot trace or debug the same program is that strace
1039becomes the parent process of the one being debugged & processes ( unlike people )
1040can have only one parent.
1041
1042
1043However the file /t1 will get big quite quickly
1044to test it telnet 127.0.0.1
1045
1046now look at what files in.telnetd execve'd
1047413 execve("/usr/sbin/in.telnetd", ["/usr/sbin/in.telnetd", "-h"], [/* 17 vars */]) = 0
1048414 execve("/bin/login", ["/bin/login", "-h", "localhost", "-p"], [/* 2 vars */]) = 0
1049 738
1050Whey it worked!.
1051
1052
1053Other hints:
1054------------
1055If the program is not very interactive ( i.e. not much keyboard input )
1056& is crashing in one architecture but not in another you can do
1057an strace of both programs under as identical a scenario as you can
1058on both architectures outputting to a file then.
1059do a diff of the two traces using the diff program
1060i.e.
1061diff output1 output2
1062& maybe you'll be able to see where the call paths differed, this
1063is possibly near the cause of the crash.
1064
1065More info
1066---------
1067Look at man pages for strace & the various syscalls
1068e.g. man strace, man alarm, man socket.
1069
1070
1071Performance Debugging
1072=====================
1073gcc is capable of compiling in profiling code just add the -p option
1074to the CFLAGS, this obviously affects program size & performance.
1075This can be used by the gprof gnu profiling tool or the
1076gcov the gnu code coverage tool ( code coverage is a means of testing
1077code quality by checking if all the code in an executable in exercised by
1078a tester ).
1079
1080
1081Using top to find out where processes are sleeping in the kernel
1082----------------------------------------------------------------
1083To do this copy the System.map from the root directory where
1084the linux kernel was built to the /boot directory on your
1085linux machine.
1086Start top
1087Now type fU<return>
1088You should see a new field called WCHAN which
1089tells you where each process is sleeping here is a typical output.
1090
1091 6:59pm up 41 min, 1 user, load average: 0.00, 0.00, 0.00
109228 processes: 27 sleeping, 1 running, 0 zombie, 0 stopped
1093CPU states: 0.0% user, 0.1% system, 0.0% nice, 99.8% idle
1094Mem: 254900K av, 45976K used, 208924K free, 0K shrd, 28636K buff
1095Swap: 0K av, 0K used, 0K free 8620K cached
1096
1097 PID USER PRI NI SIZE RSS SHARE WCHAN STAT LIB %CPU %MEM TIME COMMAND
1098 750 root 12 0 848 848 700 do_select S 0 0.1 0.3 0:00 in.telnetd
1099 767 root 16 0 1140 1140 964 R 0 0.1 0.4 0:00 top
1100 1 root 8 0 212 212 180 do_select S 0 0.0 0.0 0:00 init
1101 2 root 9 0 0 0 0 down_inte SW 0 0.0 0.0 0:00 kmcheck
1102
1103The time command
1104----------------
1105Another related command is the time command which gives you an indication
1106of where a process is spending the majority of its time.
1107e.g.
1108time ping -c 5 nc
1109outputs
1110real 0m4.054s
1111user 0m0.010s
1112sys 0m0.010s
1113 739
1114Debugging under VM 740Debugging under VM
1115================== 741==================