Debugging the linux kernel using kgdb and VirtualBox

https://fotisl.com/blog/2009/09/debugging-the-linux-kernel-using-kgdb-and-virtualbox/

September 6th, 2009Fotis

Kgdb is a source level debugger for the linux kernel. It requires two machines, one running a kernel compiled with kgdb enabled and the second one running gdb. It can be found at sourceforge and a light version has been merged into the 2.6.26 kernel. There is an article at kerneltrap which contains all the appropriate information about this light version and it’s differences from the full one. I am going to describe how you can debug a linux kernel running under VirtualBox using the kgdb-light debugger.

First of all you must define a serial port. Go to the settings of your virtual machine, then at the “Serial Ports” and enable “Port 1”. Use port number COM1, port mode ‘Host Pipe’, check ‘Create Pipe’ and enter a path, e.g. /home/fotisl/virtualbox/myvm/serial1. You can use another port number, e.g. COM2, but then you’ll have to change the device below to ttyS1, ttyS2 for COM3 etc. Furthermore, you can create the pipe yourself and not automatically using:

$ mkfifo /home/fotisl/virtualbox/myvm/serial1

1	$ mkfifo /home/fotisl/virtualbox/myvm/serial1

At your virtual machine you must have a kernel compiled with the option CONFIG_KGDB. You can find this under the “Kernel debugging” menu. I also advise you to enable the CONFIG_DEBUG_INFO to insert debug symbols.

At the host machine you only need to install socat and of course gdb. Socat is a multipurpose relay which can be found here. You should also transfer the uncompressed image of the kernel running at the vm. It can be found at the directory where you compiled the kernel and it’s name will be vmlinux.

You are now ready to start. At the host machine run:

$ socat -d -d /home/fotisl/virtualbox/myvm/serial1 pty: 2009/01/01 00:00:00 socat[12345] N opening connection to AF=1 "/home/fotisl/virtualbox/myvm/serial1" 2009/01/01 00:00:00 socat[12345] N successfully connected from local address AF=1 "\x04\b\xAB" 2009/01/01 00:00:00 socat[12345] N successfully connected via \xD0\xA7\x10 2009/01/01 00:00:00 socat[12345] N PTY is /dev/pts/4 2009/01/01 00:00:00 socat[12345] N starting data transfer loop with FDs [3,3] and [4,4]

1 2 3 4 5 6

$ socat -d -d /home/fotisl/virtualbox/myvm/serial1 pty: 2009/01/01 00:00:00 socat[12345] N opening connection to AF=1 "/home/fotisl/virtualbox/myvm/serial1" 2009/01/01 00:00:00 socat[12345] N successfully connected from local address AF=1 "\x04\b\xAB" 2009/01/01 00:00:00 socat[12345] N successfully connected via \xD0\xA7\x10 2009/01/01 00:00:00 socat[12345] N PTY is /dev/pts/4 2009/01/01 00:00:00 socat[12345] N starting data transfer loop with FDs [3,3] and [4,4]

You must note the PTY, in this case /dev/pts/4. Now fire gdb and load vmlinux. Then set the remote baud to 115200 and attach to the serial port.

$ gdb ~/vmlinux GNU gdb (GDB) 6.8.50.20090628-cvs-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i486-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... (gdb) set remotebaud 115200 (gdb) target remote /dev/pts/4 Remote debugging using /dev/pts/4

1 2 3 4 5 6 7 8 9 10 11 12 13

$ gdb ~/vmlinux GNU gdb (GDB) 6.8.50.20090628-cvs-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.  Type "show copying" and "show warranty" for details. This GDB was configured as "i486-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... (gdb) set remotebaud 115200 (gdb) target remote /dev/pts/4 Remote debugging using /dev/pts/4

Now switch to the virtual machine. You must first set the serial port that kgdb will use.

# echo ttyS0,115200 > /sys/module/kgdboc/parameters/kgdboc

1	# echo ttyS0,115200 > /sys/module/kgdboc/parameters/kgdboc

You’re ready to start debugging! When you want to break use the Alt-SysRq-G key combination or use

# echo g > /proc/sysrq-trigger

1	# echo g > /proc/sysrq-trigger

If you want to start the debugging when the kernel starts loading, append

kgdboc=ttyS0,115200 kgdbwait

1	kgdboc=ttyS0,115200 kgdbwait

to the command line parameters of the kernel. You must use this order! First you must register the I/O driver and then kgdb will be able to wait.

You can now explore the linux kernel! Warning, messing with various structures and executing code that you shouldn’t can cause kernel panics and mess up your virtual machine! But you already know that, that’s why you use virtualbox!