How to make a heap profiler

I have a new blog post at embeddedrelated.com describing heapprof, a 250-line heap profiler (C+Python) working out of the box on Linux, and easy to port/tweak.

8 comments ↓

#1 Sergey on 05.23.14 at 3:43 am

You can rewrite reduce(operator.or_, [set(block.stack) for block in blocks]) as {x for block in blocks for x in block.stack} which is both shorter and more obvious to understand IMO.

#2 Yossi Kreinin on 05.23.14 at 3:49 am

Actually, yeah – thanks, changed it.

#3 Sergey on 05.23.14 at 9:35 am

You seem to have switched the `for`s around. I agree the order you used would be more intuitive, but that's not the python syntax unfortunately:

>>> [a for a in b for b in [[1]]]
Traceback (most recent call last):
File "", line 1, in
NameError: name 'b' is not defined
>>> [a for b in [[1]] for a in b]
[1]

#4 Yossi Kreinin on 05.23.14 at 9:45 am

Um, yeah… fixed.

#5 Zeev Tarantov on 05.23.14 at 11:31 am

Why do you compile C without optimizations?

#6 Yossi Kreinin on 05.23.14 at 6:32 pm

Erm… do you ask because there's a bug exposed under -O3 or some such?

#7 Joachim Schipper on 06.10.14 at 11:11 pm

This looks cool (and inspires me to take another look at adding backtraces to our memory allocation). One question, though: isn't pulling in threads – and thus possibly a thread-aware standard library, etc. – scary?

#8 Yossi Kreinin on 06.11.14 at 2:41 am

If your app doesn't have threads you don't need to worry about it. If it does you already have a thread-aware stdlib etc. – and the code I spent on threading inside the profiler is not that much code. So I don't think it's particularly troublesome.

Leave a Comment