Enabling core dumps on Linux
written on Sat 16 August 2014
When creating a Linux application in a native language, such as C or
C++, it's obvious that eventually a bug will appear. Some are easy to
fix, but sometimes they're not. Normally you could just run
figure out how to fix it, but what about a situation when the program
runs smoothly under
gdb, but crashes when running without any
There is one feature that can be used to troubleshoot the problem, and its name is: core dumps.
A core dump is a special file that is created when the process
crashes. It's stored in binary format, but with tools like
not hard to analyze it. In fact, you don't need to analyze it at all
-- it simply stores the current crashed environment of the process in
such a way, that
gdb can be used normally, like in every other
debugging session. The only difference is that you won't be able to
execute anything from such crashed process.
You can enable core dump file generation by using the
$ ulimit -c unlimited
This will enable core dump generation for crashed applications invoked from the current shell:
$ ./crash Segmentation fault $ ls core* core.2123
2123 is the process id of the crashed application. You can load
this core dump to
gdb by providing its name in the argument list for
$ gdb crash core.2123
Gdb will notify about loading the file with messages like:
$ gdb crash core.2123 .... Core was generated by `./crash'. ....
You can use standard
bt command to build the offending stacktrace:
(gdb) bt #0 0x00007f8955ba4d67 in raise () from /usr/lib/libc.so.6 #1 0x00007f8955ba6118 in abort () from /usr/lib/libc.so.6 #2 0x00007f8955be4f93 in __libc_message () from /usr/lib/libc.so.6 #3 0x00007f8955bea88e in malloc_printerr () from /usr/lib/libc.so.6 #4 0x00007f8955beb04b in _int_free () from /usr/lib/libc.so.6 #5 0x00007f89564fdacc in std::string::assign(std::string const&) () from /usr/lib/libstdc++.so.6 #6 0x000000000044b551 in DataVisualizer::setCaptionText (this=0x1b831f0, s="ls") at /home/antek/workspace/mdhexedit/lib/plugins/interfaces/DataVisualizer.h:24 #7 0x0000000000448eda in MainInstance::getFileContextForNewFile (this=0x1aea058, captionText="ls") at ../src/ui/MainInstance.cpp:138 #8 0x0000000000432bb6 in FileMenu::loadFile (this=0x1af4e00, fileName="/bin/ls") at ../src/ui/menus/FileMenu.cpp:78
Way better than printf-debugging, don't you think?
I did everything from above, and I'm still not getting any coredumps!
Are you using
systemd? It probably depends on the distribution, but sometimes
systemd eats all the coredumps and stores them inside its own journal.
You can test if this is the case for your system by issuing a simple command:
For example, on my home system, the output of
coredumpctl looks something like
pon 2014-06-30 20:21:59 CEST 6705 1000 100 11 /personal/antek/.local/share/Steam/SteamApps/common/dota 2 beta/dota_linux wto 2014-07-01 15:49:17 CEST 1660 0 0 6 /usr/bin/python3.4 wto 2014-07-01 21:50:30 CEST 18408 1000 100 6 /home/antek/.local/share/Steam/SteamApps/common/Sid Meier's Civilization V/Civ5XP śro 2014-07-02 15:38:49 CEST 1422 0 0 6 /usr/bin/python3.4 czw 2014-07-03 10:05:46 CEST 1410 0 0 6 /usr/bin/python3.4 czw 2014-07-03 12:06:45 CEST 2515 1000 100 11 /home/antek/dev/mtf/mtf czw 2014-07-03 12:07:15 CEST 2538 1000 100 11 /home/antek/dev/mtf/mtf
You can extract the coredump you're interested in by using
example, to extract the crash from 2014-07-03 12:07:15, you can simply invoke:
$ coredumpctl dump 2538 -o core.out
This will dump this coredump directly into the
core.out file. But, consider
coredumpctl gdb command as well.