Very funny, gdb. Ve-ery funny.

December 18th, 2013

Have you ever opened a core dump with gdb, tried to print a C++ std::vector element, and got the following?

(gdb) p v[0]
You can't do that without a process to debug.

So after seeing this for years, my thoughts traveled along the path of, we could make a process out of the core dump.

No really, there used to be Unices with a program called undump that did just that. All you need to do is take the (say) ELF core dump file and generate an ELF executable file which loads the memory image saved in the core (that's actually the easier, portable part) and initializes registers to the right values (the harder, less portable part). I even wrote a limited version of undump for PowerPC once.

So we expended some effort on it at work.

And then I thought I'd just check a live C++ process (which I normally don't do, for various reasons). Let's print a vector element:

(gdb) p v[0]
Could not find operator[].
(gdb) p v.at(0)
Cannot evaluate function -- may be inlined.

Very funny, gdb. "You can't do that without a process to debug". Well, I guess you never did say that I could do that with a process to debug, now did you. Because, sure enough, I can't. Rolling on the floor, laughing. Ahem.

I suggest that we all ditch our evil C arrays and switch to slow-compiling, still-not-boundary-checked, still-not-working-in-debuggers-after-all-these-YEARS std::vector, std::array and any of the other zillion "improvements".

And gdb has these pretty printers which, if installed correctly (not easy with several gcc/STL versions around), can display std::vector – as in all of its 10000 elements, if that's how many elements it has. But they still don't let you print vec[0].member.vec2[5].member2. Sheesh!

P.S. undump could be useful for other things, say a nice sort of obfuscating scripting language compiler – Perl used to use undump for that AFAIK. And undump would in fact let you call functions in core dumps – if said functions could be, um, found by gdb. Still, ouch.

P.P.S. What gdb prints and when depends on things I do not comprehend. I failed to reproduce the reported behavior in full at home. I've seen it for years at work though.

1. Ben CraigDec 19, 2013

Debugging std::vector isn't too bad. You can dig in to the structure (p v), static_cast the first element to your data type (because it is almost certainly a void *), then you can do your array tricks on the static_casted pointer. I wish all the STL containers were as "easy" as vector. Yes, I know that all that stuff is a far cry from "p v[0]".

The real nightmare is when you want to look at any node based container. The container only has pointers to node base classes. The node base class doesn't include your element type. So you either get to cast each node to a different internal data structure type, or you get to do some funky pointer offsetting + casts. And your traversal isn't anywhere as simple as switching from "p v[0]" to "p v[1]", you have to go through a bunch of "next" calls.

2. AssafDec 25, 2013

Sheesh... MSVC debugger has been visualizing vectors for a while now.

3. Yossi KreininDec 25, 2013

gdb "visualizes" vectors as well (well, it pretty-prints them and then Eclipse and similar visualize them). The problem is evaluating expressions with vec[5] in them.

4. Michael MoserJan 7, 2014

http://www.yolinux.com/TUTORIALS/src/dbinit_stl_views-1.03.txt

should solve your problems.
Mr. Dan Marinescu wrote some macros that automate looking at _M_impl and friends.

5. Ilya KasnacheevJan 11, 2014

There is this crazy effort underway to make every linux process serializable: allow not only memory be [un]dumped but all open file descriptors status (including bringing socked in the same mode) and other environment too.

This seems to be doable. They are gradually patching the kernel to make every thing reversible.

6. MoschopsJan 11, 2014

"still-not-boundary-checked"

Oh for God's sake, please, GIVE IT A REST. If you want to use something that IS boundary checked, then do so. It's a simply modification to make yourself, or just get a library that does boundary checking, and leave those of us who don't want boundary checking in peace. Nobody's holding a gnu to your head.

7. thwestJan 11, 2014

In many standard library implementations containers are bounds checked when _DEBUG is defined.

8. NimrodJan 11, 2014

Debugging C++ code with gdb is such a frustrating and arcane experience that this issue is just one drop in a sea of problems. Essentially most people resort to printf()'s (sorry, "cerr << " if you're into this iostream nightmare).

The *real* problem is that, for some unknown reason, people don't see this as a problem and no measures are taken to improve the situation significantly.

The suggestion to use gdb macros (stl_views) is also problematic: Due to the way emacs – gdb interface is working (and you should really debug in emacs) printing more than a few tens of elements gets really slow.

9. mahtJan 11, 2014

the first thing I do with a project is say "how am I going to debug this when it fails?", you found out too late

10. MarkJan 11, 2014

I work on a parallel C++ debugger and we've made quite a few improvements to gdb, not all of which have been merged upstream yet. You could grab our patches from the website and see if they help.

Btw boost arrays are bound checked as a compile-time option, not that that helps you much now!

11. PaulJan 11, 2014

Microsoft VC++ has been letting you debug core dumps as processes for years. And VC++ has let you debug stl containers, including node containers, for years too, as it provides a debugging description language for generic data.

12. jon wJan 11, 2014

The problem is gdb, not C++. I've wanted a slick, modern debugger for Unix for the last 15 years, but not had the gumption to actually make one.

13. fagJan 11, 2014

if you want boundary checking on vectors, use the std::vector::at member function.

14. Yossi KreininJan 12, 2014

VC++ is nice but gdb supports way more platforms. In general, a lot of people tell me what I could do instead of using GNU and std::vector... oh the possibilities. The things I could do instead of programming make my head spin, for starters. Why we do what we do is such an interesting topic in itself.

15. NickJan 12, 2014

Have you tried lldb?

16. Yossi KreininJan 12, 2014

Not yet. Is it actually better? Does it support cross-debugging MIPS? Does it work with DWARF3 or does it want me to compile with clang to get the benefits?

17. NickJan 12, 2014

I can't answer that, sadly. I used only for Objective-C. Though I just tried it and lldb could not execute 'at' method. After some research I found out that those methods are inlined, so there are no real function copies exist. Same thing happens with all templates functions in standard library

18. MangerJan 12, 2014

Have you tried disabling optimizations with -O0? GDB is trying to find and run code associated to operator[]() and at(), which is impossible without an attached process (there is no code to run), or when the functions are inlined (and, with optimizations, I believe pretty much all std::vector functions are).

19. NickJan 13, 2014

Manger, I believe that you'll have to recompile stdlib.

20. NSJul 13, 2014

undump on linux:
https://code.google.com/p/undump/

As for bounds checking, why do you say that that doesn't exist? It's kind of a big selling point of vector, ever since its inception. Have you never used std::vector::at before?

21. Yossi KreininJul 13, 2014

The undump you're telling me about is the one the guy working with me wrote. It's not completely finished yet, but thanks for the tip.

As to std::vector, there's also operator[] which is what most people use, and that's not bounds checked. No thanks for the tip to use the uglier syntax; a better though still not great suggestion would be to roll one's own bounds checked vector.

22. Yossi KreininJul 13, 2014

Oh, and I used (or tried to use) at() right there in TFA.

23. argothielNov 15, 2014

What do you mean by bounds checking in []? The whole point of [] instead of at() is to create an element, if it doesn't exist yet.

24. argothielNov 15, 2014

Besides you can define -D _GLIBCXX_DEBUG flag and you won't exceed vector boundaries.

25. Yossi KreininNov 15, 2014

Create if it doesn't exist? Maybe in std::map, but not in std::vector where the behavior for out of range accesses with operator[] is undefined: http://www.cplusplus.com/reference/vector/vector/operator/

And yes, there exist non-portable build flags turning on boundary checking in operator[] – and doing a bunch of other things.

26. hongAug 4, 2015

Actually I tried with gdb 7.7.1 and compile the program with "-O0" option, there comes no problem at all with operator []:

(gdb) p xx[5]
$2 = (__gnu_cxx::__alloc_traits<std::allocator >::value_type &) @0x630a18: 2.2017165780746879

(xx is std::vector)



Post a comment