diff options
author | Thomas Huth <thuth@linux.vnet.ibm.com> | 2014-10-28 10:12:23 -0400 |
---|---|---|
committer | Martin Schwidefsky <schwidefsky@de.ibm.com> | 2014-11-18 12:22:59 -0500 |
commit | a6b42afa3fc452339e157ad5245320804cf1206f (patch) | |
tree | 6ab5ba2a6703d52e328ed89de878ceb303e2a025 /Documentation | |
parent | b19556231156ce3e58ffd677747bf3ef7890a937 (diff) |
s390/docs: Remove sections that are not related to s390
Information how to use the GCC pre-processor, objdump, strace, top, etc.
are generic and not specific to the S390 architecture, so we do not need
this information in Debugging390.txt
Signed-off-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Diffstat (limited to 'Documentation')
-rw-r--r-- | Documentation/s390/Debugging390.txt | 374 |
1 files changed, 0 insertions, 374 deletions
diff --git a/Documentation/s390/Debugging390.txt b/Documentation/s390/Debugging390.txt index 2120eec48a5c..08911b5c6b0e 100644 --- a/Documentation/s390/Debugging390.txt +++ b/Documentation/s390/Debugging390.txt | |||
@@ -26,11 +26,6 @@ The Linux for s/390 & z/Architecture Kernel Task Structure | |||
26 | Register Usage & Stackframes on Linux for s/390 & z/Architecture | 26 | Register Usage & Stackframes on Linux for s/390 & z/Architecture |
27 | A sample program with comments | 27 | A sample program with comments |
28 | Compiling programs for debugging on Linux for s/390 & z/Architecture | 28 | Compiling programs for debugging on Linux for s/390 & z/Architecture |
29 | Figuring out gcc compile errors | ||
30 | Debugging Tools | ||
31 | objdump | ||
32 | strace | ||
33 | Performance Debugging | ||
34 | Debugging under VM | 29 | Debugging under VM |
35 | s/390 & z/Architecture IO Overview | 30 | s/390 & z/Architecture IO Overview |
36 | Debugging IO on s/390 & z/Architecture under VM | 31 | Debugging IO on s/390 & z/Architecture under VM |
@@ -740,376 +735,7 @@ Debugging with optimisation has since much improved after fixing | |||
740 | some bugs, please make sure you are using gdb-5.0 or later developed | 735 | some bugs, please make sure you are using gdb-5.0 or later developed |
741 | after Nov'2000. | 736 | after Nov'2000. |
742 | 737 | ||
743 | Figuring out gcc compile errors | ||
744 | =============================== | ||
745 | If you are getting a lot of syntax errors compiling a program & the problem | ||
746 | isn't blatantly obvious from the source. | ||
747 | It often helps to just preprocess the file, this is done with the -E | ||
748 | option in gcc. | ||
749 | What this does is that it runs through the very first phase of compilation | ||
750 | ( compilation in gcc is done in several stages & gcc calls many programs to | ||
751 | achieve its end result ) with the -E option gcc just calls the gcc preprocessor (cpp). | ||
752 | The c preprocessor does the following, it joins all the files #included together | ||
753 | recursively ( #include files can #include other files ) & also the c file you wish to compile. | ||
754 | It puts a fully qualified path of the #included files in a comment & it | ||
755 | does macro expansion. | ||
756 | This is useful for debugging because | ||
757 | 1) You can double check whether the files you expect to be included are the ones | ||
758 | that are being included ( e.g. double check that you aren't going to the i386 asm directory ). | ||
759 | 2) Check that macro definitions aren't clashing with typedefs, | ||
760 | 3) Check that definitions aren't being used before they are being included. | ||
761 | 4) Helps put the line emitting the error under the microscope if it contains macros. | ||
762 | |||
763 | For convenience the Linux kernel's makefile will do preprocessing automatically for you | ||
764 | by suffixing the file you want built with .i ( instead of .o ) | ||
765 | |||
766 | e.g. | ||
767 | from the linux directory type | ||
768 | make arch/s390/kernel/signal.i | ||
769 | this will build | ||
770 | |||
771 | s390-gcc -D__KERNEL__ -I/home1/barrow/linux/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer | ||
772 | -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -E arch/s390/kernel/signal.c | ||
773 | > arch/s390/kernel/signal.i | ||
774 | |||
775 | Now look at signal.i you should see something like. | ||
776 | |||
777 | |||
778 | # 1 "/home1/barrow/linux/include/asm/types.h" 1 | ||
779 | typedef unsigned short umode_t; | ||
780 | typedef __signed__ char __s8; | ||
781 | typedef unsigned char __u8; | ||
782 | typedef __signed__ short __s16; | ||
783 | typedef unsigned short __u16; | ||
784 | |||
785 | If instead you are getting errors further down e.g. | ||
786 | unknown instruction:2515 "move.l" or better still unknown instruction:2515 | ||
787 | "Fixme not implemented yet, call Martin" you are probably are attempting to compile some code | ||
788 | meant for another architecture or code that is simply not implemented, with a fixme statement | ||
789 | stuck into the inline assembly code so that the author of the file now knows he has work to do. | ||
790 | To look at the assembly emitted by gcc just before it is about to call gas ( the gnu assembler ) | ||
791 | use the -S option. | ||
792 | Again for your convenience the Linux kernel's Makefile will hold your hand & | ||
793 | do all this donkey work for you also by building the file with the .s suffix. | ||
794 | e.g. | ||
795 | from the Linux directory type | ||
796 | make arch/s390/kernel/signal.s | ||
797 | |||
798 | s390-gcc -D__KERNEL__ -I/home1/barrow/linux/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer | ||
799 | -fno-strict-aliasing -D__SMP__ -pipe -fno-strength-reduce -S arch/s390/kernel/signal.c | ||
800 | -o arch/s390/kernel/signal.s | ||
801 | |||
802 | |||
803 | This will output something like, ( please note the constant pool & the useful comments | ||
804 | in the prologue to give you a hand at interpreting it ). | ||
805 | |||
806 | .LC54: | ||
807 | .string "misaligned (__u16 *) in __xchg\n" | ||
808 | .LC57: | ||
809 | .string "misaligned (__u32 *) in __xchg\n" | ||
810 | .L$PG1: # Pool sys_sigsuspend | ||
811 | .LC192: | ||
812 | .long -262401 | ||
813 | .LC193: | ||
814 | .long -1 | ||
815 | .LC194: | ||
816 | .long schedule-.L$PG1 | ||
817 | .LC195: | ||
818 | .long do_signal-.L$PG1 | ||
819 | .align 4 | ||
820 | .globl sys_sigsuspend | ||
821 | .type sys_sigsuspend,@function | ||
822 | sys_sigsuspend: | ||
823 | # leaf function 0 | ||
824 | # automatics 16 | ||
825 | # outgoing args 0 | ||
826 | # need frame pointer 0 | ||
827 | # call alloca 0 | ||
828 | # has varargs 0 | ||
829 | # incoming args (stack) 0 | ||
830 | # function length 168 | ||
831 | STM 8,15,32(15) | ||
832 | LR 0,15 | ||
833 | AHI 15,-112 | ||
834 | BASR 13,0 | ||
835 | .L$CO1: AHI 13,.L$PG1-.L$CO1 | ||
836 | ST 0,0(15) | ||
837 | LR 8,2 | ||
838 | N 5,.LC192-.L$PG1(13) | ||
839 | |||
840 | Adding -g to the above output makes the output even more useful | ||
841 | e.g. typing | ||
842 | make CC:="s390-gcc -g" kernel/sched.s | ||
843 | |||
844 | which compiles. | ||
845 | s390-gcc -g -D__KERNEL__ -I/home/barrow/linux-2.3/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -pipe -fno-strength-reduce -S kernel/sched.c -o kernel/sched.s | ||
846 | |||
847 | also outputs stabs ( debugger ) info, from this info you can find out the | ||
848 | offsets & sizes of various elements in structures. | ||
849 | e.g. the stab for the structure | ||
850 | struct rlimit { | ||
851 | unsigned long rlim_cur; | ||
852 | unsigned long rlim_max; | ||
853 | }; | ||
854 | is | ||
855 | .stabs "rlimit:T(151,2)=s8rlim_cur:(0,5),0,32;rlim_max:(0,5),32,32;;",128,0,0,0 | ||
856 | from this stab you can see that | ||
857 | rlimit_cur starts at bit offset 0 & is 32 bits in size | ||
858 | rlimit_max starts at bit offset 32 & is 32 bits in size. | ||
859 | |||
860 | |||
861 | Debugging Tools: | ||
862 | ================ | ||
863 | |||
864 | objdump | ||
865 | ======= | ||
866 | This is a tool with many options the most useful being ( if compiled with -g). | ||
867 | objdump --source <victim program or object file> > <victims debug listing > | ||
868 | |||
869 | |||
870 | The whole kernel can be compiled like this ( Doing this will make a 17MB kernel | ||
871 | & a 200 MB listing ) however you have to strip it before building the image | ||
872 | using the strip command to make it a more reasonable size to boot it. | ||
873 | |||
874 | A source/assembly mixed dump of the kernel can be done with the line | ||
875 | objdump --source vmlinux > vmlinux.lst | ||
876 | Also, if the file isn't compiled -g, this will output as much debugging information | ||
877 | as it can (e.g. function names). This is very slow as it spends lots | ||
878 | of time searching for debugging info. The following self explanatory line should be used | ||
879 | instead if the code isn't compiled -g, as it is much faster: | ||
880 | objdump --disassemble-all --syms vmlinux > vmlinux.lst | ||
881 | |||
882 | As hard drive space is valuable most of us use the following approach. | ||
883 | 1) Look at the emitted psw on the console to find the crash address in the kernel. | ||
884 | 2) Look at the file System.map ( in the linux directory ) produced when building | ||
885 | the kernel to find the closest address less than the current PSW to find the | ||
886 | offending function. | ||
887 | 3) use grep or similar to search the source tree looking for the source file | ||
888 | with this function if you don't know where it is. | ||
889 | 4) rebuild this object file with -g on, as an example suppose the file was | ||
890 | ( /arch/s390/kernel/signal.o ) | ||
891 | 5) Assuming the file with the erroneous function is signal.c Move to the base of the | ||
892 | Linux source tree. | ||
893 | 6) rm /arch/s390/kernel/signal.o | ||
894 | 7) make /arch/s390/kernel/signal.o | ||
895 | 8) watch the gcc command line emitted | ||
896 | 9) type it in again or alternatively cut & paste it on the console adding the -g option. | ||
897 | 10) objdump --source arch/s390/kernel/signal.o > signal.lst | ||
898 | This will output the source & the assembly intermixed, as the snippet below shows | ||
899 | This will unfortunately output addresses which aren't the same | ||
900 | as the kernel ones you should be able to get around the mental arithmetic | ||
901 | by playing with the --adjust-vma parameter to objdump. | ||
902 | |||
903 | |||
904 | |||
905 | |||
906 | static inline void spin_lock(spinlock_t *lp) | ||
907 | { | ||
908 | a0: 18 34 lr %r3,%r4 | ||
909 | a2: a7 3a 03 bc ahi %r3,956 | ||
910 | __asm__ __volatile(" lhi 1,-1\n" | ||
911 | a6: a7 18 ff ff lhi %r1,-1 | ||
912 | aa: 1f 00 slr %r0,%r0 | ||
913 | ac: ba 01 30 00 cs %r0,%r1,0(%r3) | ||
914 | b0: a7 44 ff fd jm aa <sys_sigsuspend+0x2e> | ||
915 | saveset = current->blocked; | ||
916 | b4: d2 07 f0 68 mvc 104(8,%r15),972(%r4) | ||
917 | b8: 43 cc | ||
918 | return (set->sig[0] & mask) != 0; | ||
919 | } | ||
920 | |||
921 | 6) If debugging under VM go down to that section in the document for more info. | ||
922 | |||
923 | |||
924 | I now have a tool which takes the pain out of --adjust-vma | ||
925 | & you are able to do something like | ||
926 | make /arch/s390/kernel/traps.lst | ||
927 | & it automatically generates the correctly relocated entries for | ||
928 | the text segment in traps.lst. | ||
929 | This tool is now standard in linux distro's in scripts/makelst | ||
930 | |||
931 | strace: | ||
932 | ------- | ||
933 | Q. What is it ? | ||
934 | A. It is a tool for intercepting calls to the kernel & logging them | ||
935 | to a file & on the screen. | ||
936 | |||
937 | Q. What use is it ? | ||
938 | A. You can use it to find out what files a particular program opens. | ||
939 | |||
940 | |||
941 | |||
942 | Example 1 | ||
943 | --------- | ||
944 | If you wanted to know does ping work but didn't have the source | ||
945 | strace ping -c 1 127.0.0.1 | ||
946 | & then look at the man pages for each of the syscalls below, | ||
947 | ( In fact this is sometimes easier than looking at some spaghetti | ||
948 | source which conditionally compiles for several architectures ). | ||
949 | Not everything that it throws out needs to make sense immediately. | ||
950 | |||
951 | Just looking quickly you can see that it is making up a RAW socket | ||
952 | for the ICMP protocol. | ||
953 | Doing an alarm(10) for a 10 second timeout | ||
954 | & doing a gettimeofday call before & after each read to see | ||
955 | how long the replies took, & writing some text to stdout so the user | ||
956 | has an idea what is going on. | ||
957 | |||
958 | socket(PF_INET, SOCK_RAW, IPPROTO_ICMP) = 3 | ||
959 | getuid() = 0 | ||
960 | setuid(0) = 0 | ||
961 | stat("/usr/share/locale/C/libc.cat", 0xbffff134) = -1 ENOENT (No such file or directory) | ||
962 | stat("/usr/share/locale/libc/C", 0xbffff134) = -1 ENOENT (No such file or directory) | ||
963 | stat("/usr/local/share/locale/C/libc.cat", 0xbffff134) = -1 ENOENT (No such file or directory) | ||
964 | getpid() = 353 | ||
965 | setsockopt(3, SOL_SOCKET, SO_BROADCAST, [1], 4) = 0 | ||
966 | setsockopt(3, SOL_SOCKET, SO_RCVBUF, [49152], 4) = 0 | ||
967 | fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(3, 1), ...}) = 0 | ||
968 | mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40008000 | ||
969 | ioctl(1, TCGETS, {B9600 opost isig icanon echo ...}) = 0 | ||
970 | write(1, "PING 127.0.0.1 (127.0.0.1): 56 d"..., 42PING 127.0.0.1 (127.0.0.1): 56 data bytes | ||
971 | ) = 42 | ||
972 | sigaction(SIGINT, {0x8049ba0, [], SA_RESTART}, {SIG_DFL}) = 0 | ||
973 | sigaction(SIGALRM, {0x8049600, [], SA_RESTART}, {SIG_DFL}) = 0 | ||
974 | gettimeofday({948904719, 138951}, NULL) = 0 | ||
975 | sendto(3, "\10\0D\201a\1\0\0\17#\2178\307\36"..., 64, 0, {sin_family=AF_INET, | ||
976 | sin_port=htons(0), sin_addr=inet_addr("127.0.0.1")}, 16) = 64 | ||
977 | sigaction(SIGALRM, {0x8049600, [], SA_RESTART}, {0x8049600, [], SA_RESTART}) = 0 | ||
978 | sigaction(SIGALRM, {0x8049ba0, [], SA_RESTART}, {0x8049600, [], SA_RESTART}) = 0 | ||
979 | alarm(10) = 0 | ||
980 | recvfrom(3, "E\0\0T\0005\0\0@\1|r\177\0\0\1\177"..., 192, 0, | ||
981 | {sin_family=AF_INET, sin_port=htons(50882), sin_addr=inet_addr("127.0.0.1")}, [16]) = 84 | ||
982 | gettimeofday({948904719, 160224}, NULL) = 0 | ||
983 | recvfrom(3, "E\0\0T\0006\0\0\377\1\275p\177\0"..., 192, 0, | ||
984 | {sin_family=AF_INET, sin_port=htons(50882), sin_addr=inet_addr("127.0.0.1")}, [16]) = 84 | ||
985 | gettimeofday({948904719, 166952}, NULL) = 0 | ||
986 | write(1, "64 bytes from 127.0.0.1: icmp_se"..., | ||
987 | 5764 bytes from 127.0.0.1: icmp_seq=0 ttl=255 time=28.0 ms | ||
988 | |||
989 | Example 2 | ||
990 | --------- | ||
991 | strace passwd 2>&1 | grep open | ||
992 | produces the following output | ||
993 | open("/etc/ld.so.cache", O_RDONLY) = 3 | ||
994 | open("/opt/kde/lib/libc.so.5", O_RDONLY) = -1 ENOENT (No such file or directory) | ||
995 | open("/lib/libc.so.5", O_RDONLY) = 3 | ||
996 | open("/dev", O_RDONLY) = 3 | ||
997 | open("/var/run/utmp", O_RDONLY) = 3 | ||
998 | open("/etc/passwd", O_RDONLY) = 3 | ||
999 | open("/etc/shadow", O_RDONLY) = 3 | ||
1000 | open("/etc/login.defs", O_RDONLY) = 4 | ||
1001 | open("/dev/tty", O_RDONLY) = 4 | ||
1002 | |||
1003 | The 2>&1 is done to redirect stderr to stdout & grep is then filtering this input | ||
1004 | through the pipe for each line containing the string open. | ||
1005 | |||
1006 | |||
1007 | Example 3 | ||
1008 | --------- | ||
1009 | Getting sophisticated | ||
1010 | telnetd crashes & I don't know why | ||
1011 | |||
1012 | Steps | ||
1013 | ----- | ||
1014 | 1) Replace the following line in /etc/inetd.conf | ||
1015 | telnet stream tcp nowait root /usr/sbin/in.telnetd -h | ||
1016 | with | ||
1017 | telnet stream tcp nowait root /blah | ||
1018 | |||
1019 | 2) Create the file /blah with the following contents to start tracing telnetd | ||
1020 | #!/bin/bash | ||
1021 | /usr/bin/strace -o/t1 -f /usr/sbin/in.telnetd -h | ||
1022 | 3) chmod 700 /blah to make it executable only to root | ||
1023 | 4) | ||
1024 | killall -HUP inetd | ||
1025 | or ps aux | grep inetd | ||
1026 | get inetd's process id | ||
1027 | & kill -HUP inetd to restart it. | ||
1028 | |||
1029 | Important options | ||
1030 | ----------------- | ||
1031 | -o is used to tell strace to output to a file in our case t1 in the root directory | ||
1032 | -f is to follow children i.e. | ||
1033 | e.g in our case above telnetd will start the login process & subsequently a shell like bash. | ||
1034 | You will be able to tell which is which from the process ID's listed on the left hand side | ||
1035 | of the strace output. | ||
1036 | -p<pid> will tell strace to attach to a running process, yup this can be done provided | ||
1037 | it isn't being traced or debugged already & you have enough privileges, | ||
1038 | the reason 2 processes cannot trace or debug the same program is that strace | ||
1039 | becomes the parent process of the one being debugged & processes ( unlike people ) | ||
1040 | can have only one parent. | ||
1041 | |||
1042 | |||
1043 | However the file /t1 will get big quite quickly | ||
1044 | to test it telnet 127.0.0.1 | ||
1045 | |||
1046 | now look at what files in.telnetd execve'd | ||
1047 | 413 execve("/usr/sbin/in.telnetd", ["/usr/sbin/in.telnetd", "-h"], [/* 17 vars */]) = 0 | ||
1048 | 414 execve("/bin/login", ["/bin/login", "-h", "localhost", "-p"], [/* 2 vars */]) = 0 | ||
1049 | 738 | ||
1050 | Whey it worked!. | ||
1051 | |||
1052 | |||
1053 | Other hints: | ||
1054 | ------------ | ||
1055 | If the program is not very interactive ( i.e. not much keyboard input ) | ||
1056 | & is crashing in one architecture but not in another you can do | ||
1057 | an strace of both programs under as identical a scenario as you can | ||
1058 | on both architectures outputting to a file then. | ||
1059 | do a diff of the two traces using the diff program | ||
1060 | i.e. | ||
1061 | diff output1 output2 | ||
1062 | & maybe you'll be able to see where the call paths differed, this | ||
1063 | is possibly near the cause of the crash. | ||
1064 | |||
1065 | More info | ||
1066 | --------- | ||
1067 | Look at man pages for strace & the various syscalls | ||
1068 | e.g. man strace, man alarm, man socket. | ||
1069 | |||
1070 | |||
1071 | Performance Debugging | ||
1072 | ===================== | ||
1073 | gcc is capable of compiling in profiling code just add the -p option | ||
1074 | to the CFLAGS, this obviously affects program size & performance. | ||
1075 | This can be used by the gprof gnu profiling tool or the | ||
1076 | gcov the gnu code coverage tool ( code coverage is a means of testing | ||
1077 | code quality by checking if all the code in an executable in exercised by | ||
1078 | a tester ). | ||
1079 | |||
1080 | |||
1081 | Using top to find out where processes are sleeping in the kernel | ||
1082 | ---------------------------------------------------------------- | ||
1083 | To do this copy the System.map from the root directory where | ||
1084 | the linux kernel was built to the /boot directory on your | ||
1085 | linux machine. | ||
1086 | Start top | ||
1087 | Now type fU<return> | ||
1088 | You should see a new field called WCHAN which | ||
1089 | tells you where each process is sleeping here is a typical output. | ||
1090 | |||
1091 | 6:59pm up 41 min, 1 user, load average: 0.00, 0.00, 0.00 | ||
1092 | 28 processes: 27 sleeping, 1 running, 0 zombie, 0 stopped | ||
1093 | CPU states: 0.0% user, 0.1% system, 0.0% nice, 99.8% idle | ||
1094 | Mem: 254900K av, 45976K used, 208924K free, 0K shrd, 28636K buff | ||
1095 | Swap: 0K av, 0K used, 0K free 8620K cached | ||
1096 | |||
1097 | PID USER PRI NI SIZE RSS SHARE WCHAN STAT LIB %CPU %MEM TIME COMMAND | ||
1098 | 750 root 12 0 848 848 700 do_select S 0 0.1 0.3 0:00 in.telnetd | ||
1099 | 767 root 16 0 1140 1140 964 R 0 0.1 0.4 0:00 top | ||
1100 | 1 root 8 0 212 212 180 do_select S 0 0.0 0.0 0:00 init | ||
1101 | 2 root 9 0 0 0 0 down_inte SW 0 0.0 0.0 0:00 kmcheck | ||
1102 | |||
1103 | The time command | ||
1104 | ---------------- | ||
1105 | Another related command is the time command which gives you an indication | ||
1106 | of where a process is spending the majority of its time. | ||
1107 | e.g. | ||
1108 | time ping -c 5 nc | ||
1109 | outputs | ||
1110 | real 0m4.054s | ||
1111 | user 0m0.010s | ||
1112 | sys 0m0.010s | ||
1113 | 739 | ||
1114 | Debugging under VM | 740 | Debugging under VM |
1115 | ================== | 741 | ================== |