If a tree falls in a forest, it kills Schrödinger's cat

Schrödinger used to have this quantum cat which was alive and dead at the same time as long as nobody opened the box, and it was the very act of looking at the cat that made it either alive or dead. Now, I'm not sure about this quantum stuff, but if you ask me you'd always find a dead cat upon opening the box, killed by the act of not looking. In fact, if you open any random box nobody was looking into, chances are you'll find a dead cat there. Let me give an example.

I recently chatted with a former employee of a late company I'll call Excellence (the real name was even worse). Excellence was a company with offices right across the street that kept shrinking until the recent financial crisis. It then had to simultaneously fire almost all of its remaining employees, carefully selected as their best during the previous years when other employees were continuously fired at a lower rate. Giving us a whole lot of great hires, including MV, the co-worker in this story (though he was among those who guessed where things were going and crossed the street a bit earlier).

Rumors tell that to the extent possible, Excellence used to live up to expectations created by its name. In particular, without being encouraged or forced by customers to comply to any Software Development Methodology such as the mighty and dreadful CMM, they had (as CMM puts it) not only Established, but rigorously Maintained an elaborate design, documentation and review process which preceded every coding effort. Other than praise, MV had little to say about the process, except perhaps that occasionally someone would invent something awfully complicated that made code incomprehensible, having gone just fine through the review process because of sounding too smart for anyone to object.

Now, in our latest conversation about how things were at Excellence, MV told how he once had to debug a problem in a core module of theirs, to which no changes had been made in years. There, he stumbled upon a loop searching for a value. He noticed that when the value was found, the loop wouldn't terminate – a continue instead of a break kind of thing. Since the right value tended to be found pretty early through the loop, and because it was at such a strategic place, test cases everyone was running took minutes instead of seconds to finish. Here's a dead cat in a gold-plated box for ya, and one buried quite deeply.

My own professional evolution shaped my mind in such a way that it didn't surprise me in the slightest that this slipped past the reviewer(s). What surprised me was how this slipped past the jittery bodybuilder. You see, we have this Integration Manager, one of his hobbies being bodybuilding (a detail unrelated, though not entirely, to his success in his work), and one thing he does after integrating changes is he looks at the frame rate. When the frame rate seems low, he pops up a window with the execution profile, where the program is split into about 1000 parts. If your part looks heavier than usual, or if it's something new that looks heavy compared to the rest, it'll set him off.

So I asked MV how come that the cat, long before it got dead and buried, didn't set off the jittery bodybuilder. He said they didn't have one for it to set off. They were translating between the formats of different programs. Not that performance didn't matter – they worked on quite large data sets. But to the end user, automatic translation taking hours had about the same value as automatic translation taking seconds – the alternative was manual translation taking weeks or months. So they took large-scale performance implications of their decisions into account during design reviews. Then once the code was done and tested, it was done right, so if it took N cycles to run, it was because it took N cycles to do whatever it did right.

And really, what is the chance that the code does everything right according to a detailed spec it is tested against, but there's a silly bug causing it to do it awfully slowly? If you ask me – the chance is very high, and more generally:

  • Though not looking at performance followed from a reasonable assessment of the situation,
  • Performance was bad, and bad enough to become an issue (though an unacknowledged one), when it wasn't looked at,
  • Although the system in general was certainly "looked at", apparently from more eyes and angles than "an average system", but it didn't help,
  • So either you have a jittery bodybuilder specifically and continuously eyeballing something, or that something sucks.

Of course you can save effort using jittery automated test programs. For example, we've been running a regression testing system for about a year. I recently decided to look at what's running through it, beyond the stuff it reports as errors that ought to be fixed (in this system we try to avoid false positives to the point of tolerating some of the false negatives, so it doesn't complain loudly about every possible problem). I found that:

  • It frequently ran out of disk space. It was OK for it to run out of disk space at times, but it did it way too often now. That's because its way of finding out the free space on the various partitions was obsoleted by the move of the relevant directories to network-attached storage.
  • At some of the machines, it failed to get licenses to one of the compilers it needed – perhaps because the env vars were set to good values with most users but not all, perhaps because of a compiler upgrade it didn't take into account. [It was OK for it to occasionally fail to get a license (those are scarce) - then it should have retried, and at the worst case report a license error. However, the compiler error messages it got were new to it, so it thought something just didn't compile. It then ignored the problem on otherwise good grounds.]
  • Its way of figuring out file names from module names failed for one module which was partially renamed recently (don't ask). Through an elaborate path this resulted in tolerating false negatives it really shouldn't.

And I'm not through with this thing yet, which to me exemplifies the sad truth that while you can have a cat looking at other cats to prevent them from dying, a human still has to look at that supervisor cat, or else it dies, leading to the eventual demise of the others.

If you don't look at a program, it rots. If you open a box, there's a dead cat in it. And if a tree falls in a forest and no one is around to hear it, it sucks.


#1 Antiguru on 06.02.10 at 5:53 pm

Well, if there's one thing I'm convinced is useless, it's got to be profilers. I turned around on debuggers but only because they actually became useful over time, but there's an inherent problem with profiling like using a debugger in that it seems to completely turn off people's brains.

So you get results like someone coming to you and saying that the performance bottleneck of an app is string concatenation. So you give a nomcommital grunt instead of shame them publicly in the interest of a happy workplace, and off they go rewriting string classes. Making stringbuffers for concatenation, and then they throw a meeting about it and two of the would-be alpha coders get into a long argument about whether char* or string is better to use. As if any time, ever, the performance of string concatenation mattered to someone who made a sane program.

Then the boss offers you a delicious can of soda if you find out why 'the boys' are in a big uproar the last couple days. So you go and look and see wtf is really happening. Oh! A parser. Fancy that. A parser written without a scanner! And no one knows what that is. Or why strings concatenation and parsers don't go together well except in throwaway code (which, if it were, it ought to be in Perl or some laguage that is actually good with strings). Likewise, ideas like tokens and grammars are totally alien.

After about the third time of this happening, the flaw's apparent no matter how apathetic you are. The problem is that a profiler can help with obvious mistakes like you describe but usually performance problems are not related to stupid mistakes – and the ones which are are easy for any peabrain to find anyway.

Of course we all get annoyed by quotes from CSC 101 about not optimizing or optimizing only by algorithm use, but algorithm aside there's always something that is holding you back but usually it's something like overbloated code size, disk access, oversimple blocking, etc. etc. All things you can't figure out by instruction counting, and nowadays instruction counting is never the limiting factor anyway so it becomes doubly useless. Not to mention you can't optimize everything, so hunting through each scrap is useless, especially when the people causing problems usually cause them systemwide, not in a way that shows up in the code they own. Such as if your newly minted CS grad is assigned some init process that allocates a huge amount of memory, mostly in tiny chunks via string concatenation, hopelessly fragmenting your memory before any real work gets done.

Ultimately, I think the only real thing is testing, plus having people who know what they are doing. It's just like having a rehearsal for a big play. It's guaranteed to never work without rehearsal, no matter how great the actors or the play, but with enough rehearsal even the worst play with awful actors could at least complete without a major mishap. Performance gains can also be astounding, but it's almost always a case of either a waste of time, or requiring so much in detail knowledge of every bit of C++ and your app that it is a big waste of effort of what's probably your best technical asset.

#2 raghava on 06.03.10 at 6:19 am

@antiguru: :) Seriously! Profilers (and the kind of issues they identify) are simply overrated. Agree a hundred percent about the play analogy; but it also has to be seen that the actors need to be smart enough (to a minimum extent at least) and have some presence of mind, for Murphy would always be around.

@Yossi: A good post, thanks. Once, my team was asked to look into a peculiar problem with a strange pattern, involving elements in a list. A deeper inspection of the underlying library (which was built some five years ago, no fixes done till date) revealed a dead cat: an off by one error. Why did it not stink till now? None tested it with a case which would have failed. :D

#3 njn on 06.03.10 at 12:04 pm

Profilers are useless? That's the dumbest idea I've heard in a while.

#4 Tommy on 06.03.10 at 8:10 pm

Having recently adopted two adorable kittens, I thought you were going to make the obvious point "cats in boxes die". I tried to transport my two kittens in a box, but they quickly escaped and crawled underneath of the brake pedal, so cats not in boxes almost made us all die.

Instead, what I'm getting out of this is that cats need to be looked after by humans or other cats, as long as a human is watching a cat somewhere. I'm not watching my cat chew an electrical wire right now, I hope that it doesn't die.

I completely empathize with the desire to automate everything. I'd like to automate as much of my life as possible. Clearly, the important thing to consider is that the automation process if easily made fallible, and that someone is going to notice eventually.

Problems are better noticed by a jittery bodybuilder in the office, than a run over cat on the street.

#5 Yossi Kreinin on 06.04.10 at 4:21 am

@Antiguru: if performance isn't measured, it becomes crap; I'm not saying that it can't become crap otherwise or that sometimes it's OK when it's crap, just that it will become crap if not measured. As to profilers – you generally need custom ones, timers at logically meaningful places plus counters of "things" (pixels processed/basic blocks optimized/whatever).

@raghava: that an uncovered case didn't work is perfectly legitimate and basically unavoidable, what is avoidable is covered cases that work defectively though.

@Tommy: it seems cats and death go well together. Hence the nine lives, presumably.

#6 Aaron Davies on 06.04.10 at 5:36 am

from the tao:

A well-used door needs no oil on its hinges.
A swift-flowing stream does not grow stagnant.
Neither sound nor thoughts can travel through a vacuum.
Software rots if not used.

These are great mysteries.

#7 gus3 on 06.04.10 at 7:36 pm

"Premature optimization is the root of all evil." But after a certain point, a well-designed profiler can expose contradictory assumptions between a subroutine and its caller.

#8 Amir Barak on 06.07.10 at 6:38 am

Of course sometimes dead cats were simply put on top of leaky drainpipes; especially it seems when it isn't your code and sometimes because it IS your code [or mine :P]

I would always hesitate before touching anything that's been running for years [profiled badly or not]…

#9 Yossi Kreinin on 06.07.10 at 7:59 am

Well, I'm conservative enough to hesitate to touch anything that's been running for weeks if not days, it's just that sometimes you have to.

#10 Antiguru on 06.18.10 at 1:04 am

Not using a profiler doesn't mean never measure performance. It just means that you don't get sucked a false idea of how optimization is supposed to work which seems to be the case for everyone I personally know using them.

A argument I got into recently was against virtual methods being slow. My idea was to use pure virtual methods to accomplish some tasks in a sort of base node object. Mr scruples just couldn't allow the word virtual to pass his fingers, let alone a pure virtual.

He even made up a case and executed it a billion times or so to prove his point. It was a little faster, not not a lot. But I knew for a fact it was actually faster in a real world app. The virtual lookup is static data. Instead there's this dumb 'control' object that is an object with its own function pointer. So it's not static but there for every single listener.

So I make a realworld case and tahdah! It's obvious the virtual method is faster in the case we actually care about. and the margin wider than it had been in the BS case.

The reason is as the bits fly none of this crap is in cache any more, and what's worse is it's not together in memory and it has some bizarre indirection, so you wind up doing 2-3 cache mishes every single lookup when things are not in chache. Plus, it takes up more cache and pretty much everything is a node, so this was also degrading the whole system by helping choke the cache out.

Of course not everyone just knows off the top of their head, but you are going to solve 100X more perfomance issues by knowing than by just measuring and coming to an obvious conclusion.

#11 Yossi Kreinin on 06.18.10 at 3:56 am

Of course you need some mental model of why things could take the time they take to make sense of measurements; and, when you write optimized code, obviously you're not doing measurement-driven iteration (change, measure, change, measure…) but rather have some overall plan of an optimized implementation, having weighed alternatives in your head rather than done measurements for N implementations.

All I'm saying is that on top of that, when you have an app coded by reasonable people, where performance isn't then continuously measured, it becomes crap.

#12 6502 on 07.03.10 at 1:50 pm

One phrase I found (sadly) true is that if everything works then it's just because you didn't look closely enough.

It's *so* common in complex systems to look for a bug and then after some hunting you think you nailed it down and say "AH!… here it is!! when we introduced that change this part of the program wasn't updated correctly and .. blah blah blah" and indeed you found a true genuine bug!

But it's another one, not the one you were chasing…

dead cats are everywhere…


PS: For code efficiency is even worse… especially when people starts doing "optimizations" without using a decent (i.e. passive) profiler first. I've seen a major code rewrite because of a single operation that was O(n^2) and taking 99.9% of cpu time and could have been coded as O(1) with minimal impact on the rest of the program.
After the change computation in one use case went down from 25 minutes to 3 seconds (and those were mostly the update of the progress bar that was added to inform users that the system wasn't indeed dead).

#13 noop on 01.12.11 at 10:34 am

Yes, profilers are useless in most cases. Problem is that good programmer will use correct and efficient method from the start and bad programmer can't improve by only using a profiler or come other tool, instead of reading some good books.
Existence of profilers and optimizing compilers is being used as excuse for writing sloppy code. Today many students are taught: "write working code first, think about speed later( usually never )".
P.S. Sorry for bad english.

#14 Oisín on 06.12.12 at 3:22 pm

"Existence of profilers and optimizing compilers is being used as excuse for writing sloppy code. Today many students are taught: “write working code first, think about speed later( usually never )”."

People are taught this for a good reason. If you try to write "correct and efficient" code from the start as you recommend, you are making your task artificially more difficult, and therefore increasing the likelihood of making errors.

Instead, a more robust approach would be to focus on writing correct code first, with some decent test coverage, before reviewing your design and code and refactoring/optimising.

This is not to say that you should purposely write "sloppy" code or reject all thoughts of performance, just that the focus should not be on writing optimal code in the first pass. To presume that you can write correct and well-optimised code on the first attempt is optimistic and rather dangerous.

Going back to profilers, I don't buy the idea that they only help people to solve trivial optimisation problems.
A more important use is to justify where optimisation is worthwhile.

For example, imagine a programmer comes to a team meeting claiming that some important feature was currently taking O(n^3) time when it could be O(n log n) with another algorithm, and that we should replace the algorithm immediately.
Someone raises their hand and asks how much time this would save. Nobody answers, so the programmer does a profiling run and discovers that the "slow" algorithm currently takes 50ms, which is dwarfed by the slow network queries that are performed immediately afterwards.

Like any tool, profilers can be misused, but they certainly have some value.

#15 free gg hack on 05.15.19 at 4:11 pm

Enjoyed reading through this, very good stuff, thankyou .

#16 fortnite aimbot download on 05.16.19 at 12:41 pm

I like this site because so much useful stuff on here : D.

#17 aimbot fortnite on 05.16.19 at 4:35 pm

I conceive this web site holds some real superb information for everyone : D.

#18 nonsense diamond on 05.17.19 at 6:49 am

I am glad to be one of the visitors on this great website (:, appreciate it for posting .

#19 fallout 76 cheats on 05.17.19 at 10:16 am

Morning, i really think i will be back to your website

#20 Tatyana Tajudeen on 05.17.19 at 12:49 pm

yosefk.com is good, solid content. I just now forwarded this on 5/17/2019 to a coworker who has been doing a little research of her own on this subject. To say thanks, she just bought me lunch! So, let me express my gratitude by saying: Cheers for the drink!

#21 red dead redemption 2 digital key resale on 05.17.19 at 3:26 pm

Ni hao, here from yanex, me enjoyng this, will come back again.

#22 redline v3.0 on 05.17.19 at 6:29 pm

This i like. Cheers!

#23 chaturbate hack cheat engine 2018 on 05.18.19 at 7:55 am

Hi, i really think i will be back to your page

#24 led ryggsäck on 05.18.19 at 2:47 pm

Respect to website author , some wonderful entropy.

#25 mining simulator 2019 on 05.19.19 at 6:47 am

I simply must tell you that you have an excellent and unique web that I kinda enjoyed reading.

#26 smutstone on 05.20.19 at 11:26 am

Me enjoying, will read more. Cheers!

#27 redline v3.0 on 05.21.19 at 6:56 am

I truly enjoy looking through on this web site , it holds superb content .

#28 free fire hack version unlimited diamond on 05.21.19 at 4:10 pm

bing took me here. Cheers!

#29 nonsense diamond on 05.22.19 at 6:00 pm

I conceive this web site holds some real superb information for everyone : D.

#30 krunker aimbot on 05.23.19 at 6:19 am

Found this on bing and I’m happy I did. Well written post.

#31 bitcoin adder v.1.3.00 free download on 05.23.19 at 9:57 am

Yeah bookmaking this wasn’t a risky decision outstanding post! .

#32 vn hax on 05.23.19 at 6:41 pm

stays on topic and states valid points. Thank you.

#33 eternity.cc v9 on 05.24.19 at 7:29 am

I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.

#34 ispoofer pogo activate seriale on 05.24.19 at 5:54 pm

Ha, here from google, this is what i was searching for.

#35 cheats for hempire game on 05.26.19 at 6:20 am

I like this website its a master peace ! Glad I found this on google .

#36 iobit uninstaller 7.5 key on 05.26.19 at 9:06 am

You got yourself a new rader.

#37 smart defrag 6.2 serial key on 05.26.19 at 3:26 pm

I have interest in this, xexe.

#38 resetter epson l1110 on 05.26.19 at 6:03 pm

I like this article, some useful stuff on here : D.

#39 sims 4 seasons code free on 05.27.19 at 7:21 am

I am not rattling great with English but I get hold this really easygoing to read .

#40 rust hacks on 05.27.19 at 7:54 pm

Enjoyed examining this, very good stuff, thanks .

#41 strucid hacks on 05.28.19 at 10:12 am

yahoo bring me here. Thanks!

#42 expressvpn key on 05.28.19 at 7:15 pm

Some truly fine posts on this web site , appreciate it for contribution.

#43 ispoofer activation key on 05.29.19 at 8:28 am

I dugg some of you post as I thought they were very beneficial invaluable

#44 gamefly free trial on 05.29.19 at 9:52 am

Hello there, I discovered your web site by means of Google at the same time as searching for a related topic,
your web site came up, it looks great. I have bookmarked it in my google bookmarks.

Hi there, just became aware of your blog via Google, and located that
it is really informative. I am going to be careful for brussels.
I will appreciate for those who continue this in future.

Many folks might be benefited from your writing. Cheers!

#45 aimbot free download fortnite on 05.29.19 at 12:26 pm

I conceive this web site holds some real superb information for everyone : D.

#46 vn hax on 05.30.19 at 6:06 am

I like this site, some useful stuff on here : D.

#47 how to get help in windows 10 on 05.30.19 at 9:22 am

Hmm it appears like your site ate my first
comment (it was super long) so I guess I'll just sum it up what I submitted and say, I'm
thoroughly enjoying your blog. I too am an aspiring blog writer but I'm still new to the whole
thing. Do you have any tips and hints for novice blog writers?
I'd definitely appreciate it.

#48 gamefly free trial on 05.31.19 at 3:40 am

I constantly emailed this web site post page to all my associates, for the reason that if like to read it
then my contacts will too.

#49 xbox one mods free download on 05.31.19 at 12:40 pm

I really enjoy examining on this page , it has got good goodies .

#50 fortnite aimbot download on 05.31.19 at 3:25 pm

I dugg some of you post as I thought they were very beneficial invaluable

#51 gamefly free trial on 06.01.19 at 4:33 am

It's an amazing piece of writing in support of all the online
people; they will get benefit from it I am sure.

#52 gamefly free trial on 06.01.19 at 11:55 am

Highly descriptive article, I liked that a lot. Will there
be a part 2?

#53 mpl pro on 06.01.19 at 6:22 pm

Just wanna input on few general things, The website layout is perfect, the articles is very superb : D.

#54 hacks counter blox script on 06.02.19 at 6:28 am

Respect to website author , some wonderful entropy.

#55 gamefly free trial on 06.02.19 at 10:28 pm

I absolutely love your website.. Excellent colors & theme.
Did you make this web site yourself? Please reply
back as I'm hoping to create my very own site and would like to learn where you
got this from or exactly what the theme is called.

#56 Bennie Macgowan on 06.03.19 at 4:09 pm

In my view, yosefk.com does a excellent job of covering subject matter of this kind! Even if often intentionally polemic, the posts are in the main well researched and stimulating.

#57 gamefly free trial on 06.03.19 at 6:29 pm

Hiya very nice web site!! Man .. Excellent .. Amazing ..
I'll bookmark your blog and take the feeds additionally?
I am satisfied to seek out so many helpful information right
here within the submit, we want work out more strategies on this regard, thanks
for sharing. . . . . .

#58 gamefly free trial on 06.03.19 at 11:20 pm

Hey there! Would you mind if I share your blog with my facebook group?
There's a lot of people that I think would really appreciate your content.
Please let me know. Cheers

#59 gamefly free trial on 06.04.19 at 2:00 pm

Hmm it seems like your site ate my first comment
(it was extremely long) so I guess I'll just sum it up what I wrote and say, I'm thoroughly enjoying your blog.
I too am an aspiring blog blogger but I'm still new to the whole thing.
Do you have any helpful hints for first-time blog writers?
I'd genuinely appreciate it.

#60 quest bars cheap on 06.14.19 at 5:27 pm

Hey! I realize this is sort of off-topic however I needed to ask.

Does managing a well-established website such as yours
take a lot of work? I am brand new to writing a blog however I do write in my journal everyday.
I'd like to start a blog so I will be able to share my personal
experience and thoughts online. Please let me know if you have any recommendations
or tips for brand new aspiring blog owners.

#61 noob vs pro vs hacker vs god on 06.17.19 at 1:25 am

Ha, here from bing, this is what i was browsing for.

#62 proxo key on 06.19.19 at 10:06 am

Respect to website author , some wonderful entropy.

#63 vn hax pubg mobile on 06.20.19 at 6:56 pm

Enjoyed reading through this, very good stuff, thankyou .

#64 nonsense diamond on 06.21.19 at 8:09 am

Some truly wow posts on this web site , appreciate it for contribution.

#65 css3 2019 on 06.22.19 at 4:06 am

This site was… how do you say it? Relevant!! Finally I've found something which helped me. Thanks a lot!

#66 quest bars cheap on 06.23.19 at 8:36 am

I blog often and I really appreciate your information. The article has truly peaked my interest.

I am going to take a note of your blog and keep checking for new information about once a week.
I opted in for your RSS feed as well.

#67 Best Wordpress Plugins on 06.23.19 at 1:10 pm

The very next time I read a blog, I hope that it does not disappoint me as much as this one. I mean, I know it was my choice to read through, nonetheless I really believed you would have something interesting to talk about. All I hear is a bunch of whining about something you can fix if you weren't too busy seeking attention.

#68 star valor cheats on 06.23.19 at 5:34 pm

I’m impressed, I have to admit. Genuinely rarely should i encounter a weblog that’s both educative and entertaining, and let me tell you, you may have hit the nail about the head. Your idea is outstanding; the problem is an element that insufficient persons are speaking intelligently about. I am delighted we came across this during my look for something with this.

#69 katekowalsky on 06.23.19 at 6:58 pm

Good day! I could have sworn I’ve visited your blog before but after browsing through a few of the posts I realized it’s new to me. Nonetheless, I’m definitely pleased I found it and I’ll be book-marking it and checking back often!

#70 gx tool apk on 06.24.19 at 3:37 pm

I really enjoy examining on this blog , it has got fine content .

#71 free online Q & A on 06.25.19 at 5:30 am

Enjoyed examining this, very good stuff, thanks .

#72 geometry dash 2.11 download on 06.25.19 at 8:19 pm

I was looking at some of your articles on this site and I believe this internet site is really instructive! Keep on posting .

#73 krunker aimbot on 06.26.19 at 6:58 am

I like this site, because so much useful stuff on here : D.

Leave a Comment