Which of those would you like me to write?

"You have 59 posts and 103 drafts", says WordPress, and I guess it's right. SELECT COUNT() FROM table" doesn't lie, and then I've always had about 2 unpublished drafts per published post.

Let's try this: I'll give you this list of drafts and you'll tell me which of these you want me to write. I have a bunch, they're all fresh enough for me to still want to write them – I still remember what I meant to say – but help me prioritize.

So, here goes:

  • FPGAs vs cellular automata: cellular automata are one example of trying to model computation using local rules, "because the laws of nature are local" or what-not. But real life is not like a game of life, and long wires are everywhere – from long axons to the trans-Atlantic cables we humans laid out right after learning how to do so. I'd write about how FPGAs, which are popular with people who like "locality" and "big flexible computing grids" and other such ideas, support complicated routing and not just simple connections to neighbors. (Which is why FPGAs are a real platform and not a pipe dream.)
  • Mircoarchitecture vs macroarchitecture. I love this one, because it sounds like a somewhat original or at least not a very widely appreciated idea. Basically in hardware, there are two different ways to attack problems. Examples of micro vs macro are: out-of-order versus VLIW; SIMT vs SIMD; hardware prefetchers vs DMA engines; caches vs explicitly managed local memories; and software-driven processors vs FPGAs. Basically it's an implicit vs explicit thing – you can make hardware that cleverly solves the problem in a way which is opaque to software, or you can expose "bare" resources that software can manage in order to solve the problem itself. There are a whole lot of implications which I think aren't obvious, at least as a whole and perhaps separately – for example, one approach is friendlier to context switching than the other. This could be a good read for someone on a small team designing hardware – I could convince him to do macro and accept the drawbacks; micro is for Intel.
  • Efficiency: shortcuts and highways. To get from A to B, you travel less – that's a shortcut – or travel more simply – that's a highway. (Highways are necessarily "simpler" than other paths – wider, straighter, less junctions – and they're expensive to build, so not every path can be a highway.) It appears to be a good metaphor for optimization and accelerator hardware, in many ways. For example, getting to the highway can be a bottleneck – similarly, transferring and organizing data for accelerators/optimized functions. There are other similarities. This should, in particular, be a nice read for someone good at algorithmic optimization (shortcuts) who is curious about "brute force" optimization (highways).
  • "Don't just blame the author of this manual" – this is a quote from an actual manual that I like; the context is that the manual bluntly tells why a feature may likely do something different from what you think it does, and what you should do about it. Basically this spec is outrageous if you think of it as a contract – not much is promised to you - but very good as communication – you understand exactly what the designers were doing and what to expect as a result. The distinction between "specs as contracts" and "specs as communication" is an interesting one in general.
  • Leibnitz management: I mention "the best of all possible worlds" together with the efficient market hypothesis, which is what some ways of putting the "efficiency" argument on its head remind me of. For instance, the market is efficient and therefore, if we spend $1M on software or equipment, they must be worth the money (the option of us being suckers and the vendor being sustained by suckers is ignored). Similarly, if you propose an improvement, you're told that it's not going to work since if it did, others would have done it already because of said "efficiency". And other similar notions, all  popular with management.
  • "No rules" is a rule, and not a simple one: I guess I don't like rules, so I tend to think we should have less of them. It recently occurred to me that what we'd really want is not less rules but simpler rules and that it's not the same thing. For example, if you have no rules about noise, then one has a right to make noise (good) but no right for silence (bad), which is a trade-off that some like and others don't – but on top of that, it creates ugly complications so isn't a simplification compared to having a rule against noise (for example, I can make noise near your property to devalue it). Similarly, giving someone "a right to configure" takes someone else's "right to certainty" – be it config files or different device screen sizes or whatever – also a trade-off. Someone's right to check in code without tests takes someone else's right to count on checked-out code to work. Someone's right to pass parameters of any type takes someone else's right to count on things being of a certain type. So, not only choosing the rules, but choosing the amount of rules is a hairy trade-off. Which goes against of my intuition that "less rules is good" , all else being equal.
  • Does correctness result from design or testing? – two claims. 1: correctness always results from testing, never from design; if it wasn't tested, it doesn't work, and moreover, contains some sort of "conceptual" flaw and not just a typo. 2: however, the very property of testability always results from sound design. If it's sufficiently badly designed, no amount of testing can salvage it, because good testing is whitebox testing or at least "gray box" testing and bad design is impenetrable to the mind, so it's a black box.
  • Testing is parallelizable, analysis isn't – a complementary idea (perhaps I'd merge them). Suppose you have $10M to throw at "quality" – finding bugs. You could massively test your program, or you could thoroughly analyze it – formal proofs or human proofreading or something. Testing you can parallelize: you can buy hardware, hire QA people and so on. The insight is that you can't really parallelize analysis: to even tell that two parts can be analyzed separately, a lot of analysis is required, and then frequently things truly can't be analyzed separately, because if you understand the fact that listing a huge directory is horribly slow on your filesystem, this understanding is worthless in isolation from the fact that some piece of your software creates huge directories. So analysis can't happen fast even if you're willing to pay money – you have to pay in time. Two things follow: when something is shipped fast and works, we can conclude that it's made to work through testing and not analysis; and, pushing for testing is characteristic of the private/commercial sector where money is cheaper, whereas pushing for analysis is characteristic of the public/academic sector where time is cheaper.
  • Buying code takes more knowledge then building it - because when you build it yourself, you can afford to acquire much of the knowledge as you go, but when you're making a purchasing decision and you lack some of the required knowledge, then you'll buy the wrong thing and will then acquire the knowledge through having things not work – much too late. I'd give A LOT of real-life examples from my own real life; it's quite the problem, I believe. (There's frequently no way around buying code, especially if you're making hardware, but I think it's still an interesting angle on "buy vs build".)
  • Make your code serial and your data scalar: I claim that things that don't work that way are not debuggable and most people have a hard time with them. For example, type systems (C++, Haskell, Scala, even CLOS), vector languages (APL, Matlab before they had a fast for loop), Prolog (even though I think solvers are a great idea), Makefiles (even though I think DSLs are a great idea), lazy evaluation (Haskell).

There's more but let's see what about these.

43 comments ↓

#1 RRR on 03.23.12 at 9:49 am

Top 2 preferences:
1. FPGAs vs cellular automata
2. Mircoarchitecture vs macroarchitecture

#2 Andrew Jenner on 03.23.12 at 10:08 am

I'd like to see the first two as well – they're things I've thought about in the past and I'm interested to see what your take on them is.

#3 Frank Adrian on 03.23.12 at 10:12 am

In order…

1. Microarchitecture vs. macroarchitecture
2. "Don't just blame the author of this manual"
3. Efficiency: Shortcuts and highways
4. Make your code serial and your data scalar

#4 Sergey on 03.23.12 at 10:13 am

+1 for "Leibnitz management" and another for "buying code..".

WRT "simple" rules, I think about similar matters in terms of adding strong invariants — if checkins are conditional on tests passing, then the passing test are an invariant. A related direction in programming would be avoiding inheritance or any other kind of type dispatching. Or having a rule of only putting locks in variables / attrs that have 'lock' in their name and 'evt' for events, so you can find all locks and events in the program with a simple text search.

Setting up voting on Google Moderator could be a good option too.

#5 Marshall on 03.23.12 at 10:23 am

1) Microarchitecture vs macroarchitecture.

2) Make your code serial and your data scalar.

3) Correctness and testing.

Thank you very much, by the way, both for your great writing and your willingness to please!

#6 Dirkjan Ochtman on 03.23.12 at 11:32 am

1. Buy vs build
2. Simple rules

#7 Nathan on 03.23.12 at 12:09 pm

microarchitecture vs macroarchitecture
buy vs build

#8 jonathan on 03.23.12 at 12:11 pm

1. Mircoarchitecture vs macroarchitecture

2. Buying code takes more knowledge then building it – I like real life examples.

3. Make your code serial and your data scalar – Mainly because I don't understand your description, and that makes me curious.

#9 Ancorehraq sis on 03.23.12 at 12:51 pm

FPGAs vs cellular automata for informativeness
Leibnitz management for sheer spite

#10 PSO on 03.23.12 at 2:20 pm

* “Don’t just blame the author of this manual”
* Buying code takes more knowledge then building it
* “No rules” is a rule, and not a simple one

#11 Tolomea on 03.23.12 at 2:43 pm

All of the above :P

Probably "Make your code serial and your data scalar" the most.

#12 saurabh on 03.23.12 at 3:43 pm

first two over the rest

#13 Pawel on 03.23.12 at 3:52 pm

Really curious about the FPGAs.

#14 Aristotle Pagaltzis on 03.23.12 at 4:14 pm

I will read all of them. But these are the ones I am specifically interested in – in this order:

Make your code serial and your data scalar
Efficiency: shortcuts and highways
Mircoarchitecture vs macroarchitecture
“Don’t just blame the author of this manual”
Leibnitz management
FPGAs vs cellular automata

#15 gus3 on 03.23.12 at 6:06 pm

No rules vs. simple rules. It's the only description I read all the way through, so that shows you how much it held my interest. :)

#16 Yura Bronnikov on 03.23.12 at 10:14 pm

1. Simple rules.
2. Serial code, scalar data.
3. Leibniz (no t).
4. Micro vs macro.

#17 Z.T. on 03.23.12 at 10:16 pm

I would like to read all of them. I especially want to read stories about ways things fail in your experience (re: testing, haskell, buying code, etc.)

#18 Yossi Kreinin on 03.24.12 at 12:44 am

Thanks for your responses, and in particular, it's nice to know that hardware is a popular topic (though I always wonder whether my writing about that stuff gets too detailed or instead too incomprehensible, because I never know what is obvious and widely known and what is obscure in these areas.)

#19 Tony Finch on 03.24.12 at 3:23 am

Blame the manual: is this an argument for there being a decent amount of explanatory rationale in a spec?

No rules vs simple rules: this is just the liberal dilemma, balancing rights and responsibilities, fairness and freedom.

Buying vs building and serial + scalar sound interesting.

#20 Leandro Lucarella on 03.24.12 at 4:23 am

Efficiency: shortcuts and highways
“Don’t just blame the author of this manual”
“No rules” is a rule, and not a simple one

#21 Stephen Schulze on 03.24.12 at 4:40 am

I am especially curious about
1. Mircoarchitecture vs macroarchitecture.
2. Make your code serial and your data scalar
because the first is a thing I am always musing about, and the second because I wonder if you actually mean "obfuscated", i.e. implicit, structure and what you would do about it.

But I will surely read any of the proposed articles.

#22 Andrea on 03.24.12 at 5:39 am

I second Marshall's choice.

To encourage you I shall confess I have found your blog a must-read for undergrad students, having been one myself.

#23 Jari Mustonen on 03.24.12 at 11:22 am

I would really like to hear your argument for serial code and scalar data.

#24 Dan on 03.24.12 at 9:33 pm

Someone I respect very much once said, "You can't test quality into a product."

Therefore, I vote for "Does correctness result from design or testing?"

My bias might be showing. I'm curious what your position is (I would love it if you took a hard stand in one camp or the other, regardless of which one. Guarantee you'll get strong opinions on both sides, watch out reddit.)

#25 Assaf on 03.25.12 at 3:12 am

Buying vs. Building code is a real conundrum. Would like to read your thoughts about it.

#26 ssshhsh on 03.25.12 at 1:23 pm

micro vs macro sounds very good

#27 maht on 03.26.12 at 3:18 am

1. FPGAs vs cellular automata
2. Mircoarchitecture vs macroarchitecture

"I always wonder whether my writing about that stuff gets too detailed or instead too incomprehensible, because I never know what is obvious and widely known and what is obscure in these areas."

Another example of where does one erect barriers, in this case to detail or you might end up explaining how a transistor works.

#28 el Mīko on 03.26.12 at 4:58 pm

+1 for "Make your code serial and your data scalar"

#29 jaredeh on 03.27.12 at 12:57 am

code serial and data scalar please

#30 David on 03.29.12 at 7:35 am

FPGAs vs cellular automata
Mircoarchitecture vs macroarchitecture
Make your code serial and your data scalar

#31 J on 03.29.12 at 8:25 am

Design v. Testing sounds interesting, although I think I'm outvoted here.

#32 Ilya on 03.29.12 at 9:27 am

My top three:

1) "Don’t just blame the author of this manual"
2) “No rules” is a rule, and not a simple one
3) Buying code takes more knowledge then building it

#33 tim on 03.30.12 at 2:07 am

1) Make your code serial and your data scalar
2) “Don’t just blame the author of this manual”
…but I too would read any of them

#34 gild on 03.31.12 at 10:27 pm

1) Micro vs. Macro
2) Make your code serial and your data scalar

#35 Dmitri on 04.04.12 at 6:35 am

Serial code, scalar data.

#36 trombone on 04.23.12 at 6:48 am

I really like reading your blog and would happily read all of these. Giving a priority to the list of options, it would be approximately:

1) Testing is parallelizable, analysis isn’t.
2) Does correctness result from design or testing?
3) Make your code serial and your data scalar.
4) Efficiency: shortcuts and highways
5) Mircoarchitecture vs macroarchitecture.
6) “No rules” is a rule, and not a simple one.
7) FPGAs vs cellular automata.
8) “Don’t just blame the author of this manual”.
9) Buying code takes more knowledge then building it.
10) Leibnitz management.

Thanks for all the hard work!

#37 Dan on 04.24.12 at 5:00 am

Buy vs. Build
Correctness from design or testing

#38 Me on 04.26.12 at 12:49 pm

1. Buying code takes more knowledge then building it
2. Testing is parallelizable, analysis isn’t

#39 Yossi Kreinin on 04.27.12 at 8:12 am

Thanks again for your votes; I'm currently working on macro vs micro which seems very popular – it's coming out long, I might split it into parts perhaps.

#40 10098 on 05.08.12 at 5:40 pm

Make your code serial and your data scalar, please!

#41 Evan on 05.09.12 at 11:35 am

A late vote for: "Make your code serial and your data scalar."

Perhaps after macro vs. micro?

#42 Nick Zivkovic on 05.11.12 at 8:59 am

Another late vote:

In case you decide to complete another draft, I'd love to see "Efficiency: shortcuts and highways"

#43 stipan on 05.21.12 at 4:28 am

I am dying to know what serial code is and how to pack everything in scalar data.

Leave a Comment