Don't ask if a monorepo is good for you – ask if you're good enough for a monorepo

This is inspired by Dan Luu's post on the advantages of a single big repository over many small ones. That post is fairly old, and I confess that I'm hardly up to date on the state of tooling, both for managing multiple repos and for dealing with one big one. But I'm going to make an argument which I think mostly works regardless of the state of tooling on any given day:

  • Monorepo is great if you're really good, but absolutely terrible if you're not that good.
  • Multiple repos, on the other hand, are passable for everyone – they're never great, but they're never truly terrible, either.

If you agree with the above, the choice is up to your personal philosophy. To me, for instance, it's a no-brainer – I'll choose the passable thing which successfully withstands contact with apathetic mediocrity over the greater thing which falls apart upon such contact in a heartbeat.

You might be different – you might believe in Good – and then you'll choose a monorepo, like Google, the ultimate force for Good in technology (which is why they safeguard your personal data; you wouldn't want someone evil to have it – luckily, Google can do no evil.) And I'm almost not kidding: the superpower which lets Google maintain the grassroots bureaucracy which I find necessary to make monorepos work well is actually the same trait making you sufficiently delusional to chant, or at least to have chanted "Don't Be Evil" entirely seriously. I don't have that. I am, to a first approximation, evil. Worse is Better.

But that's me – I'm not saying you/your org are Not So Good, or Evil. I'm only saying that you should be open to the possibility, and that I don't see the implications of being Not So Good discussed as much as they deserve.

Why are monorepos terrible if you're not that good? Three reasons:

  1. Branching in
  2. Modularity out
  3. Tooling strained

Let's discuss them in some detail.

Branching: getting forked by your worst programmer

In a Good team, you don't have multiple concurrent branches from which actual product deliveries are produced, and/or where most people get to maintain these branches simultaneously for a long time. And you certainly can't have branching due to outright atrocities, like someone adding a feature by killing a feature – for example, making the app work on Android, but destroying the ability to build for iOS in the process.

But in a not-so-good team… you get the idea.

What do you do when you have a branch working on Android and another branch working on iOS and you have deliveries on both platforms? You postpone the merge, and keep the fork. For how long do you postpone the merge? For as long as is necessary for the dumbass who caused the fork to fix their handiwork, in parallel with delivering more features (which likely results in digging a deeper hole to climb out of afterwards.) And the dumbass might take months, years, or forever.

The question then becomes, what was forked?

In a multi-repo world, the repo maintained by the team with the dumbass on it got forked. In a monorepo world, the entire code base got forked, and the entire org is now held hostage by the dumbass. And you might think that this will result in a lot of pressure to fix the problem, and you'd be wrong, for the same reasons that high murder rates don't cure themselves by people putting pressure on whomever to lower them to some equilibrium level common to all human societies.

Some places have higher than average murder rates, and some places have have higher than average fork rates. And I argue that a lot of places have fork rates which combine into a complete disaster with a monorepo. And you might not even realize how bad the fork rate is at your place, because multiple repos largely shield you from the consequences. Or, more tragically, you might not realize how bad your fork rate is because your monorepo is in its first couple of years, and you're sowing what you'll reap in its next couple of years, when you'll have more code, more deliveries and more dumbasses.

With multiple repos, if you have your shit under control, and your repos have a single release branch with a single timeline, all you have to do is to test against both of the dumbass's branches. But with a monorepo, you need to maintain your code in 2 branches, with a growing share of everybody else's code morphing incompatibly in those branches, simply because they exist. And very soon it will be more than 2 because there's more than a single dumbass, and good luck to you.

Modularity: demoted from a norm to an ideal

Norms are mundane, but they are what is. Ideals are lofty, but they are merely what should be (and typically isn't.) If you want to actually have something, you don't want it to be an ideal, like altruism – you want it to be a norm, like wiping one's ass. If something is demoted from ass-wiping to altruism, that something will scarcely be found in the wild.

With multiple repos, modularity is the norm. It's not a must - you technically can have a repo depending on umpteen other repos. But your teammates expect to be able to work with their repo with a minimal set of dependencies. They don't like to have to clone lots of other repos, and to then worry about their versions (in part because the tooling support for this might be less than great).

In fact, a common multi-repo failure mode is that people expect too few dependencies and make too many repos which are too small to host a useful self-contained system. Note that this failure mode is not lethal. It kinda sucks to have this over-modularity with benefits of independence which turn out to be imaginary upon a closer look, and to have people treat what essentially are internal APIs with way too much reverence, just because two modules which are extremely tightly coupled conceptually are independent technically, in terms of cloning/building/testing. But it doesn't kill you.

With a monorepo, modularity is a mere ideal. Everybody clones the whole thing. You're not supposed to add gratuitous dependencies, but it's very easy to add such a dependency in terms of cloning, building and versioning, and nobody objects to the dependency being added the way they would if they needed to clone more repos.

Of course in a Good team, needless dependencies would be weeded out in code reviews, and a Culture would evolve over time avoiding needless dependencies. In a not-so-good team, your monorepo will grow into a single giant ball of circular dependencies. Note that adding dependencies is infinitely easier than untangling them, much like forking is easier than merging, with the difference that the gut-felt urgency to merge ("I can't maintain all your damned branches any longer!!") is far greater and far more backed by simple self-interest than the urgency to improve the dependency structure.

Tooling: is yours better than the standard?

This part might age worse than the others, and might not be particularly up to date even now – what "standard" tools are capable of changes over time. But generally speaking, a growing monorepo is likely to outgrow the standard version management tools and methods, as well as other tools and methods dealing with your revision controlled code.

Google used to have a FUSE driver to avoid copying hundreds of millions of source lines at a time, and instead getting the files on demand, when a directory is cd'd into. Facebook used to hack on hg to make it fast on its large monorepo. Maybe already today, or some day, a growing number of off-the-shelf tools will scale to infinite monorepos without such investments. But it sounds reasonable that there will always be tools and workflows which you will struggle to make work with a large monorepo (starting with some script doing find/grep.)

With a bunch of small monorepos, you work with a small overall number of source files in your working directory, so you don't need to tell your tools, "don't try to deal with the whole thing – instead only search this subset, or use this index etc. etc." And you have tools these days which kinda sorta let you manage the revisions of multiple repositories (for instance, there's Google's Repo.) And I think the result is very, very far from a great experience potentially afforded by a large monorepo. But it also never breaks down as badly as a large monorepo outgrowing the abilities of tools, as well as the ability of your local toolsmiths to find creative workarounds for these growth pains.


Don't ask if a monorepo is good for you – ask if you're good enough for a monorepo. Personally, I don't have the guts to bet on the supply of Goodness in a given org to remain sufficiently large over time to consistently avert the potential disasters of monorepos. But that's just my personal outlook; if you want to compliment me, don't call me "smart," and definitely don't call me "good" – I know my limits in these areas, and I take far more pride in knowing these limits than in the limits themselves; so, to compliment me, call me "pragmatic." Yet a culture worthy of a monorepo absolutely can exist – just make sure yours actually is one of those, and don't mistake your ideals for your norms.

Patents: how and why to get them

I'm going to discuss 3 very basic things about patents:

  • Why it's good for you to get them;
  • Why it might be bad for your employer (and why they don't care);
  • How to get a patent for your idea (doesn't matter which.)

Some of my points are a bit naughty. But I maintain that they're based in fact and fairly widely known. So well-known, in fact, that I'm surprised to have never read it somewhere else.

My explanation is that the hatred of patents in the tech world is such that nothing except "HATE! HATE! HATE!" can be said on the subject in polite society. In this atmosphere, "Patents: how and why to get them" reads like "Humans: how and why to cook them."

If you can make yourself read this human-cooking manual, however, I think you'll find both amusing and useful things. I have more experience with patents than I've ever asked for, having worked on this stuff with lawyers from the smallest law firms to the largest ones, including lawyers who personally handled the most famous lawsuits for the most famous tech clients. I'm not an authority on patents, but I have good stories.

What patents give you

Some companies pay you money per patent. But it's rarely enough to make it worth your while, unless it's all you're doing. Patents look good on your CV, but reactions might be negative as well (you might appear "overqualified," "an expert in an unrelated field," etc.)

What's the one thing a patent undeniably buys you? A right to legally and publicly discuss your work – which you often can't get in any other way. This is not a side-effect of patent law, but its whole stated point. Patent law prompts companies to publish their ideas, in exchange for a time-limited monopoly right to use the ideas.

Note that publishing ideas in patents is easy, and the benefit for the author is certain. But getting and enforcing a monopoly for said ideas is not easy, so the benefit for the proprietor is not at all certain. Here's why.

What patents give (and don't give) your employer

Some problems with patents are so obvious that even patent lawyers will honestly discuss them with their clients:

  • When you submit a patent application, it becomes public forever, even if it's rejected. You will have paid legal fees with the end result of granting competitors access to your ideas.
  • If you sue for patent infringement, your patent might be invalidated as a result. It's like a rejected patent application, but with at least $1 million more in legal fees.

But there's another, potentially far bigger problem, that patent lawyers will rarely mention, let alone admit its extent:

  • You don't get monopoly rights to everything you file in the patent application. You publish a "spec" and "claims." The monopoly is granted only for the claims – perhaps in a reduced form relatively to the original patent application, due to feedback from the examiner. Yet the entire spec, much of it not covered by the claims, becomes public.

So what's the big deal, you might ask? The spec describes some device or method. The claims describe the supposedly new ideas used in this device or method. All you have to do is write a spec such that nothing of value is disclosed that is not covered by the claims.

However, in reality, the published spec is often quite close to the actual spec used by engineers, with all the details. That's simply the path of least resistance:

  • Patent lawyers don't know which claims will be rejected by the examiner. (If they knew, you wouldn't have a heap of rejected applications, nor patents invalidated in courts.) They file relatively broad claims, and then change the claims to address challenges by the examiner, until a patent is granted. The catch is that you can only base your new claims on details included in the originally filed spec – the spec can never be altered. Thus a detailed, complete spec maximizes the chances to get some patent out of the filing – covering 90% or 10% of the spec, depending.
  • More prosaically, if we don't file the actual spec but instead write a new one tailored to the claims, who's gonna do it? Neither the engineer nor the lawyer necessarily has the ability to do it, and surely neither has any interest in doing it. Much better to take existing documents and do the minimal necessary translation from English to legalese.

Ultimately, there's a conflict of interest between your employer and their patent lawyer, and a surprisingly perfect alignment of interests between the lawyer and yourself:

  • The lawyer wants to publish as many details as possible – to maximize the chance of getting a patent, and to avoid extra work;
  • The engineer also wants to publish as much as possible – to make his ideas known to the fullest extent, and to avoid extra work;
  • The employer/shareholder wants to publish as little as possible – but has no simple, reliable way to incentivize anyone to push in this direction (though of course some are much better at this than others.)

Funnily enough, this too is largely in line with the lawmaker's stated intent – prompting companies to publish ideas instead of keeping them secret. But why do companies file patents?

The answer is that patents are never read – they're counted. More precisely, a company's goal is to acquire enough patents so that they can only be counted – but not read and understood in a reasonable amount of time.

If you have too many patents to read and understand (hundreds, thousands or more), then investors and competitors alike assume you "own your domain" – you can counter-attack if sued. You're as well-defended legally as you can possibly be. But if you have few patents, someone might read and understand most of them – and create a narrative about some legal weakness. Such narratives are bad for the stock price.

This situation must be avoided. And that's all there is to it – at least in the computing industry. And I know it might sound too dismissive to be convincing. But the fact is that the content of patents is just too complex to drive business decisions. The feasible thing for a decision-maker is to pick the bucket to put you in, out of "no patents, some patents, a shitton of patents." For more information, see the seminal work "Pulling Decisions out of One's Ass: Fast and Slow," keeping in mind that decision-makers have a lot of decisions to make, so they must be Fast.

Why filing patents isn't a crime on par with cannibalism

Considering the above, I don't think that a product company employee filing patents pollutes the tech environment as badly as people believe.

Product companies file patents largely for self-defense. Some occasionally attack startups, but how many startups were destroyed by a patent lawsuit vs the number of those destroyed by a badly managed acquisition (with the original investors doing just fine)? And there are examples of big companies buying startups already attacked by a lawsuit filed by a bigger product company, confident that between two big companies, the legal result will be a stalemate. Thus for a big company genuinely fearing your product, it's much safer to buy you than sue you and have you bought by a big competitor.

The real trouble is patent trolls, who cannot be counter-attacked. But the only way a product company's patents will land in a troll's hands is if the company goes bankrupt and sells the patents. Well, guess what – in these cases, other product companies are eager to outbid the trolls. For example, when MIPS Technologies was sold to Imagination for ~$60 million many years ago, its patents, sold separately, fetched ~$500 million from some CPU cartel involving various big name CPU companies. Alternatively, a failing company can turn into a troll and sue a successful product company (MicroUnity comes to mind.)

Thus patents of failing product companies result in a weird form of socialism, where profit is spread more evenly between investors, with losers getting a chunk of the winners' profits. I don't think this chunk is nearly large enough on average to substantially reduce the incentive to work hard for the win, which is supposedly "the" trouble with laws subsidizing losers.

My point is that patent trolls and product companies seem to live in largely parallel universes. There are patents filed with the intention to be used by a patent troll, and there are patents filed by product companies, and the latter cause far less damage.

How to get a patent

I've lost count of the number of times I've heard the words "The Black Swan." It rather aggrieves me, but you gotta hand it to Taleb. Everyone is trying to pollute our language by needlessly coining catchphrases in a quest to be memorable, but he succeeded more than most. Surely I wouldn't hear this nonsense as often if he called the book "The Unforeseen Event."

Getting patents is a lot like branding. The trick is to call old things new names.

Why does it take a patent lawsuit and at least $1 million in legal fees to find out if a patent really is a patent – or to see it invalidated by the court? Because searching prior art is hard. "Prior art" includes everything published prior to the patent – older patents, academic papers, and everything else, really. Strictly speaking, you never know if you're done.

How does the patent office examiner examine prior art, at a cost much lower than $1 million? Some equivalent of quick googling. The input of search engines is words and short phrases. If you use words and phrases which are uncommon in your domain, the search will come up blank, or it will find things so obviously unrelated to your work that even a patent examiner will get it.

If you're extending the concept of a thread, don't call the result "extended threads", call it "hypercontexts." If you're calculating a histogram, call it a "distribution estimator." And so on. Again, I know this sounds too dismissive of the system to be believable. Well, try it. File a patent application full of "distribution estimators" and another one written in plain English. See which gets approved more smoothly.

Note that you might be tempted to conduct a prior art search yourself before filing the patent application, as a matter of due diligence. Yet some lawyers actually recommend against it, since if you do find prior art, you're now willfully infringing on it, and should cease and desist. My advice is to come up with a bunch of Black Swans/Distribution Estimators describing your idea, and pick the ones with the fewest Google results (patent search and otherwise).

And don't actually read any patent you accidentally find – don't willfully infringe, it's illegal. Just count them. Patents are never read, only counted – sounds familiar?

The other very important thing – which you mostly get to worry about in smaller companies – is to get the right kind of lawyer. Patent lawyers are fallen engineers, with engineering degrees, and sometimes actual engineering experience. The underlying engineer who's morphed into a lawyer ought to have specialized in your domain. No compromise is acceptable here. If you're doing optics, don't work with a guy who did chip design, and if you do chip design, don't work with a guy who did optics.

It doesn't matter if the lawyer is a Partner ($900/hour), an Associate ($450/hour), or some lesser life form in the law firm. It doesn't matter whether the firm is the biggest name in the industry or completely unknown. What matters is engineering knowledge. Don't expect a patent lawyer to honestly tell you he doesn't know your domain. He'll always accept the work, and you'll pay $truckload/hour trying to explain the most basic things to him, and failing. You need to actively ask about his education and experience.


Like most annoying things in life, patents aren't evil as much as they're absurd. Use them to your advantage.

Things want to work, not punish errors

For better or worse, things want to work.

Consider driving at night on unlit, curvy mountain roads, at a speed about twice the limit, zigzagging between cars, including oncoming ones. Obviously dangerous, and yet many do this, and survive. How?

  • Roads and cars are built with big safety margins
  • Other drivers don't want to die and help you get through
  • Practice makes perfect, so you get good at this bad thing

The road, the car, you, other drivers, and their cars all want this to work. So for a long while, it does, until it finally doesn't. I know 3-4 people who drive like this habitually. At least 2 of them totaled cars. All think they're excellent drivers. All have high IQs, making you wonder just what this renowned benchmark of human brains really tells us.

Now consider a terribly managed project with an insane deadline, and a team and budget too small. All too often, this too works out. How?

  • Unless it physically cannot exist, a solution wants you to find it. You carve out a piece and the next piece suggests itself. Even if management fails to think how the pieces fit together, the pieces often come out such that they can be made to fit with modest extra effort.
  • And then the people who make the pieces want them to fit. Even if the process is totally mismanaged, many people will talk to each other and find out what to do to make parts work together.
  • The project was approved because a customer was persuaded. At this point, the customer wants the project to succeed. A little bit of schedule slippage will not make them change their minds, nor will a somewhat less impressive result. More slack for you.
  • The vendor, too wants the project to succeed, and will tolerate a little bit of budget overrun. More slack.
  • Most often, when things fail, they fail visibly. It's as if things wanted you to see that they fail, so that you fix them.

The fact is that by cutting features, having a few non-terminal bugs, and being somewhat late and over budget, most projects can be salvaged. In fact, when they say that "most projects fail," the PMI (*) defines "failure" as being a bit late or over budget. If "failure" is defined as outright cancellation, I conjecture that most projects "succeed."

Which projects are least likely to be canceled? In other words, where is being late, over budget and off the original spec most tolerable? Obviously, when the overall delivered value is the highest, both in absolute terms and relatively to the cost. In other words, reality punishes bad management the least in the most impactful cases.

What is the biggest problem with bad management? Same as crazy driving: risk. The problem in both cases is you risk high-cost, low-probability events. It's terrible things that tend not to happen. And people are pretty bad at learning from mistakes they never had to pay for.

Wannabe racecar drivers fail to learn from driving into risky situations which their own eyes tell them are risky. For managers, learning is harder – the risks accumulated through bad management are abstract, instead of viscerally scary. In fact, a lot of the risks are never understood by management, or even fully reported. There's just too much risk to sweep under various rugs to make it all ingrained in institutional memory.

In fact, it's even worse, because risk-taking is actually rewarding as long as the downside doesn't materialize. The crazy driver gets there 10 minutes earlier. Similarly, non-obviously hazardous management often delivers at an obviously small cost. And while driving is not actually competitive, except in the inflamed minds of the zigzagging few, most projects are delivered in very competitive environments indeed. And competition can make even small rewards for risk decisive – as it can with any other smallish factor large enough to make a difference between victory and defeat.

Things want to work more than they want to punish us for our errors. The punishment may be very cruel and unusual alright, but it's rare. It seems that the universe, at least The Universe of Deliverables, is Beckerian. It delivers optimal punishment for rational agents correctly estimating probabilities. Sadly, humans are bad at probability.

And thus crazy drivers and bad managers alike (often the same people, BTW) march from one insane adventure to the next, gaining more and more confidence in their brilliance.

(*) PMI (The Project Management Institute) is a con, where they sell you "PMBOK" (Project Management Body of Knowledge, a thick book you can use as a monitor stand) and "PMP" (Project Management Professional, a certification required by PMI's conscious or unwitting accomplices in dark corners of the industry.) A variety of more elaborate cons targeted at narrower audiences incorporate PMI's core body of cargo cult practices.

Hiring (self-driving algos, HLL compiler research)

OK, so 2 things:

1. If you send me a CV and they're hired to work on self-driving algos – machine vision/learning/mapping/navigation, I'll pay you a shitton of money. (Details over email.) These teams want CS/math/physics/similar degree with great grades, and they want programming ability. They'll hire quite a lot of people.

2. The position below is for my team and if you refer a CV, I cannot pay you a shitton of money. But:

We're developing an array language that we want to efficiently compile to our in-house accelerators (multiple target architectures, you can think of it as "compiling to a DSP/GPU/FPGA.")

Of recent public efforts, perhaps Halide is the closest relative (we're compiling AOT instead of processing a graph of C++ objects constructed at run time, but I'm guessing the work done at the back-end is somewhat similar.) What we have now is already beating hand-optimized code in our C dialects on some programs, but it's still a "blue sky" effort in that we're not sure exactly how far it will go (in terms of the share of production programs where it can replace our C dialects.)

As usual, we aren't looking for someone with experience in exactly this sort of thing (here especially it'd be hopeless since there are few compiler writers and most of them work on lower-level languages.) Historically, the people who enjoy this kind of work have a background in what I broadly call (mislabel?) "discrete math" -  formal methods, theory of computation, board game AI, even cryptography, basically anywhere where you have clever algorithms in a discrete space that can be shown to work every time. (Heavyweight counter-examples missing one of "clever", "discrete" or "every time" – OSes, rendering, or NNs. This of course is not to say that experience in any of these is disqualifying, just that they're different.)

I think of it as a gig combining depth that people expect from academic work with compensation that people expect from industry work. If you're interested, email me (

All positions are in Jerusalem.

Fun won't get it done

OK, published at 3:30 AM. That's a first!

So. Got something you want to do over the coarse of a year? Here's a motivation woefully insufficient to pull it off:

  • It's fun!

What could give you enough drive to finish the job? Anything with a reward in the future, once you're done:

  • Millions of fans will adore me.
  • It will be the ugliest thing on the planet.
  • I will finally understand quantum neural rockets.
  • We will see who the loser is, Todd!
  • I will help humanity.
  • I will destroy humanity.

It doesn't matter how noble or ignoble your goal is. What matters is delaying gratification. Because even your favorite thing in the world will have shitty bits if you chew on a big enough chunk of it. A few months or years worth of work are always a big enough chunk, so there will be shitty bits. Unfortunately, it's also the minimum-sized chunk to do anything of significance.

This is where many brilliant talents drown. Having known the joy of true inspiration, it's hard to settle for less, which you must to have any impact. Meanwhile, their thicker peers happily butcher task after task. Before you know it, these tasks add up to an impactful result.

In hindsight, I was really lucky in that I chose a profession for money instead of love. Why? Stamina. Money is a reward in the future that lets you ignore the shittier bits of the present.

Loving every moment of it, on the other hand, carries you until that moment which you hate, and then you need a new sort of fuel. Believe me, I know. I love drawing and animation, and you won't believe how many times I started and stopped doing it.

But the animation teacher who taught me 3D said he was happy to put textures on toilet seat models when he started out. That's the kind of appetite you need – and very few people naturally feel that sort of attraction to toilet seats. You need a big reward in the future, like "I'm going to become a pro," to pull it off.

But I don't want to become a pro. I don't want to work in the Israeli animation market where there's scarcely a feature film made. I don't even want to work for a big overseas animation studio. I want to make something, erm, something beautiful that I love, which is a piece of shit of a goal.

Because you know where I made most progress picking up actual skills? In an evening animation school, where I had a perfectly good goal: survive. It's good because it's a simple, binary thing which doesn't give a rat's ass about your mood. You either drop out or you don't. But "something I love" is fluid, and depends a lot on the mood. And when you hate this thing you're making, as you sometimes will, it's hard to imagine loving it later.

Conversely, imagining how I don't drop out is easy. This is what I was imagining when sculpting this bust, which 90% of the time I hated with a passion because it looked like crap. But I thought, "I'm not quitting, I'm not quitting, I'm not quitting, hey, I get the point of re-topology in Mudbox, I'm not quitting, I'm not quitting, hey, I guess I see what the specular map does, I'm not quitting… Guess I'm done!"

And now let's talk about beauty for a moment.

I'm a programmer. I like to think that I'm not the thickest, butcherest programmer, in that I understand the role of beauty in it. For the trained eye, programs can be beautiful as much as math, physics or chess, and a beautiful program is better for business than the needlessly uglier program. (Ever tried pitching the value of beauty to someone businessy? Loads of fun.)

But you know why beauty is your enemy? Because it sucks the fun out of things. How? Because you're making this thing and chances are, it's not beautiful according to your own standard. The trap is, your taste for beauty is usually ahead of your creative ability. In any area, and then in any sub-area of that area, ad infinitum, you can tell ugly from beautiful long before you can make something beautiful yourself. And even if you can satisfy your own taste, often the final thing is beautiful, but not the states it goes through.

So the passionate, sensitive soul is hit twice:

  1. You're driven by fun and inspiration because you've once experienced it and now you covet it.
  2. Your sense of beauty, frustrated by the state of your creation, kills all the fun – that very fun which you insist must be your only fuel.

Life is easier if you want a yacht. I think you can buy a decent one for $300K, and certainly for $1M. Now all you need to do is make that money, doing doesn't matter what – imagining that yacht will help you do anything well! If you want beauty, however, I do not envy you.

How do I cope with my desire for beauty? The first step is acknowledging the problem, which I do. The fact is that my worst failures in programming came when I insisted on beauty the most. The second step is shunning beauty as a goal, and making it into a means and a side-effect.

I need a program doing at least X, taking at most Y seconds, at a date not later than Z. I'll keep ugliness to a minimum because ugly programs work badly. And if it comes out particularly nicely, that's great. But beauty is not a goal, and enjoying the beauty of this program as I write it is not why I write it.

And if you think it's true for commercial work but not open source software, look at, I dunno, Linux. Read some Torvalds:

Realistically, every single release, most of it is just driver work. Which is kind of boring in the sense there is nothing fundamentally interesting in a driver, it's just support for yet another chipset or something, and at the same time that's kind of the bread and butter of the kernel. More than half of the kernel is just drivers, and so all the big exciting smart things we do, in the end it pales when compared to all the work we just do to support new hardware.

Boring bits. Boring bits that must be done to make something of value.

Does this transfer to art or poetry or any of those things whose whole point is beauty? Well, yeah, I think it does, because no, beauty is not the whole point:

  • The most important thing about a drawing is that it's done. Now it exists, and people can see it, and you can make another one. Practice. They will not come out very well if they don't come out.
  • Often people like your subject. There's a continuum between "it's beautiful in a way that words cannot convey" and "I love how this song expresses my favorite political philosophy." To the extent that a work of art tells a story, or even sets up a mood, its beauty does become a means to an end.
  • Just because the end result is beautiful to the observer, and even if that's the only point, doesn't mean every step making it was an orgy of beauty for whomever made it. Part of what goes into it is boring, technical work.

So here, too I'm trying to make beauty a non-goal. Instead my goals are "make a point" and "keep going," and you try to add beauty, or remove ugliness, as you go.

For example, I didn't do a graduation project in the evening school, but I animated a short on my own in the same timeframe, and I published it, even though it's not the beautiful thing I always dreamed about making. And I'm not sure anyone gets the joke except me. (I'm not sure I get it anymore, either.)

Now my goal is "make another one." It's a good goal, because it's easy to imagine making another one. It's proper delayed gratification.

And if you've enjoyed programming 20 years ago and are trying to reignite the passion, I suggest that you find a goal as worthy for you as "fun" or "beauty", but as clear and binary as a yacht. And you can settle for less worthy, but not for less clear and binary. Because everything they told you about "extrinsic motivation" being inferior to "intrinsic motivation" is one big lie. And this lie will fall apart the moment you sink your teeth into a bunch of shit, as will always happen if you're trying to accomplish anything.

Follow me on Twitter to receive pearls of wisdom such as the following sample:


The habitat of hardware bugs

I wrote a post on about hardware bugs - places where they're rarely to be found, places which they inhabit in large quantities, and why these insects flourish in some places more than others.

It's one of these things that I wish I was told when I started to fiddle with this shit – that while a chip is monolithic from a manufacturing point of view, from the logical spec angle it's a hodgepodge of components made and sold by different parties with very different goals and habits, tied together well enough to be marketable, but not enough to make it coherent from a low-level programmer's point of view. In fact, it's the job of the few low-level programmers to hide the idiosyncratic and buggy parts so as to present a coherent picture to the many higher-level programmers - the ones whose mental well-being is an economically significant goal.


Looking for senior IT/DevOps people

I wouldn't spam you with these job offers if didn't work :-) So, we're looking for senior IT people to work at our Jerusalem offices – managers and hands-on people alike. We have rapid growth, "Big Data" (it definitely is crash Excel - in fact, at one point it was close to physically crashing through the floor due to the storage servers' weight, but luckily that's been handled), "HPC" (biggish server farms, distributed build & tests, etc.), and many other buzzwords [1]. I don't know where IT ends and DevOps starts but I guess a good candidate could have either in their CV, so there.

If you have qualified friends looking for a challenging, well-paying job at a fun place, send their CVs, the sooner the better – we're in a hurry (rapid growth!), so early birds are more likely to get the can of worms. As always, "challenging" is a downside as much as an upside (a place where IT means Exchange, SAP and little else might pay very well for a more predictable and less demanding job.)

We value experience in building and maintaining non-trivial systems, and technical reasoning (X happens because of Y, Z is most efficient if you use it to do W, etc.) We also value experience in higher-level areas such as management and purchasing, and business reasoning (don't hook X and Y together since their vendors compete and will sabotage the project, Z beats W in terms of total cost of ownership, etc.) We do kinda lean towards thinking of technical aptitude as a cornerstone on top of which solid higher-level expertise is built. (We've seen managers snowed by vendors, reports, etc., which is a perennial problem in tech at large and isn't restricted to IT.)

If you'd like to hear more details, please email

[1] what we don't have is a heavy-duty web site/application, which might make the position less relevant for some.

A layman's view of the economy

First of all, I proudly present a 2-minute short that I animated!

…And the same thing on YouTube, in case one loads better than the other:

One thing I learned making the film is that my Russian accent colors not only my words, but any noise coming out of my mouth. So I'm not the most versatile voice actor.

Anyway, we certainly have a debt crisis, and easy credit policies keep producing still more debt. I don't think interest rates have ever stayed so low for so long, everywhere.

Economists argue both for and against debt expansion [1], as they argue about everything.

My own take is as simple as my sparse knowledge ought to make it:

  • Unprecedented conditions produce unprecedented outcomes.
  • Booms are usually gradual, and busts are sudden.

No unusual boom has gradually arisen from unusual monetary policy, and it's been a while. But something unusual ought to happen in unusual conditions! Thus one expects a sudden, unusual bust down the road.

That's it. It's like a physicist's proof [2] that one's attractiveness peaks at some distance from the observer. At the distances of zero and infinity, visual attractiveness is zero (you can't see anything.) Therefore, attractiveness as a function of distance has a maximum somewhere in between. True, kinda, and it didn't take a lot of insight into the nature of attractiveness – much like my peak debt proof doesn't require an understanding of the economy [3].

Will today's "Brexit" trigger the global downturn predicted by Yossi Kreinin's Rule of Unprecedented Conditions? Probably not by itself. I think it's a symptom more than a cause [4], and the big bad thing comes later.

In the meanwhile, here's to hoping that my little film (started when "Grexit" was a thing, completed just in time for Brexit) was funnier than the average forward from grandma [5].

Happy Brexit! And if you follow people on Twitter, there's a strong case for following me as well.

[1] Bibliography: Nobody Understands Debt except Krugman; Does Krugman Understand Debt?

[2] I think a particular famous physicist said it, but I forget who.

[3] …and I can't say I have any understanding of the economy. That said, I've owed and paid off a lot of debt, and got to negotiate with many bankers. And I can tell you that "debt is money we owe to ourselves", Krugman's catchphrase, feels unconvincing to creditors – as many people and whole nations have found out.

[4] In fact, I just got an email from an asset manager saying that it's good for the UK in the longer run, elevating Brexit from a symptom to a cure. But he didn't say "good for everyone", and then I'm not sure his crystal ball is better than yours or mine.

[5] I linked to /r/forwardsfromgrandma since, regardless of the politics of either its members or their grandmas, I ought to give credit for the brilliant term – it's definitely funny because it's true. I've watched many relatives acquire the habit of forwarding various wingnut stuff as they age. Most frighteningly, my own urge to email such things gets harder to resist every year. I can sense my own ongoing grandmafication; between you and me, an animated short about debt might be a part of "it." Scary, scary stuff.

Evil tip: avoid "easy" things

Now you see that evil will always triumph, because good is dumb.

Dark Helmet

Evildoers live longer and feel better.


My writing has recently prompted an anonymous commenter to declare that people like me are what's wrong with the world. Oh joy! – finally, after all these years of doing evil, some recognition! Excited, I decided to share one of my battle-tested evil tips, which never ever failed evil me.

Don't work on "easy" things

An easy thing is a heads they win, tails you lose situation. Failing at easy things is shameful; succeeding is unremarkable. Much better to work on hard things – heads you win, tails they lose. Failure is unfortunate but expected; success makes you a hero.

Treat this seriously, because it snowballs. The guy working on the "hard" thing gets a lot of help and resources, making it easier to succeed – and easier to move on to the next "hard" thing. The guy doing the "easy" tasks gets no help, so it's harder to succeed.

Quotation marks all over the place, because of course what counts is perception, not how hard or easy it really is. The worst thing to work on is the hard one that's perceived as easy – the hard "easy" thing. The best thing is the easy "hard" one. My years-long preference for low-level programming results, in part, from its reputation of a very hard field, when in practice, it takes a little knowledge and a lot of discipline – but not any outstanding skills.

(Why then do many people fear low-level programming? Only because of how hard it bites you the first few times. People who felt that pain and recoiled respect those who've moved past it and reached productivity. Now you know why people take shit from the likes of Ken Thompson and Linus Torvalds, and then beg for more.)

The point where this gets really evil is not when a heroic doer of hard things decides to behave like a Torvalds. That's more stupid than evil. You'll get away with being a Torvalds, but it always costs some goodwill and hence, ultimately, money. So the goal-oriented evildoer always tries to be on their best behavior.

No, the point where this gets really evil is when you let them fail. When they come to you thinking that it's easy, and you know it's actually very hard, and you turn them down, and you let them fail a few times, and you wait until they come back with a readjusted attitude - that's evil.

Here, the evildoer needs to strike a delicate balance, keeping in mind The Evildoer's Golden Rule:

  • You can only sustain that much do-gooding; however,
  • Your environment can only take that much evildoing, and you need your environment.

Here's the rule applied to our situation:

  • Working on the hard "easy" thing – all trouble, no credit – is going to be terrible for you. You'll get a taste of a do-gooder's short, miserable life.
  • However, if this thing is so important that a failure would endanger the org, maybe you should be the do-gooder and save them from their misconceptions at your own expense. Maybe. And maybe not. Be sure to think about it.

The upshot is, sometimes the evildoer gets to be the do-gooder, but you should know that it's hazardous to your health.

Making easy things into hard ones: the postponing gambit

Sometimes you can't weasel out of doing something "easy." An interesting gambit for these cases is to postpone the easy thing until it becomes urgent. This accomplishes two things at a time:

  • Urgent things automatically become harder, a person in a hurry more important. The later it's done, the easier it is to get help (while retaining the status of "the" hero in the center of it all who made it happen.)
  • Under time pressure, the scope shrinks, making the formerly "easy" and now officially "hard" thing genuinely easier. This is particularly useful for the really disgusting, but unavoidable work.

But it is a gambit, because postponing things until they become urgent is openly evil. (Avoiding easy things is not – why, it's patriotic and heroic to look for the harder work!) To get away with postponing, you need an excuse:

  • other supposedly urgent work;
  • whoever needing this thing not having reminded you;
  • or even you having sincerely underestimated the difficulty and hence, regrettably, having postponed it too much – you're so sorry. (This last excuse has the drawback of you having to admit an error. But to the extent that urgency will make the scope smaller, the error will become smaller, too.)

One thing you want to prevent is people learning to remind you earlier. The way to accomplish it is being very nice when they come late. If people feel punished for reminding too late, they'll come earlier next time, and in a vengeful mood, so with more needless tasks. But if they're late and you eagerly "try to do the best under the circumstances", not only do you put yourself under the spotlight as a patriotic hero, you move the forgetful culprit out of the spotlight. So they'll form a rosy memory of the incident, and not learn the value of coming earlier – precisely what we want.

One thing making the postponing gambit relatively safe is that management is shocked by the very thought of people playing it, as can be seen in the following real-life conversation:

Babbling management consultant: A lot of organizations have a problem where they only work on urgent things at the expense of important, but less urgent ones.

Low-ranking evildoer manager (in a momentary lapse of reason): Why, of course! I actually postpone things to get priority around here.

Higher-ranking manager (in disbelief): You aren't serious, of course.

Low-ranking evildoer (apparently still out to lunch): I am.

Higher-ranking manager (firmly): I know you aren't.

Low-ranking evildoer finally shuts his mouth.

See? Sometimes they won't believe it if you say to their face. So they're unlikely to suspect you. (Do people reporting to me play the postponing gambit? Sometimes they do, and I don't resent them for it; their priorities aren't mine. But at the worst case, you should expect a lot of resentment – it's practically high treason – so you should have plausible deniability.)


To a very large extent, your productivity is a result of what you choose to work on. Keep things perceived as easy out of that list. When you can't, postponing an "easy" thing can make it both "harder" and smaller.

Happy evildoing, and follow me on Twitter!


Looking for a functional safety/ISO 26262 expert (anywhere on the globe)

Unlike most positions mentioned here, this one includes the possibility of working remotely (certainly from Europe and I think from elsewhere, too), with occasional visits to Jerusalem.

Functional safety experts with automotive experience are generally rare and in demand, meaning that

  • they're probably gainfully employed, and
  • I'm not counting on one of them reading this blog.

However, I imagine that a friend of a safety expert might be among my readers. If you're that reader, you can tell your friend the safety expert that we're very eager to hire them, and will go a long way to make an attractive proposition.

We particularly value experience at the chip/ASIC side of things (translating ISO 26262 requirements to actionable guidelines on hardening the design in question, together with a verification methodology and the required safety documentation.) We also value recommendations from designers who had to follow the expert's guidelines, as well as experience in presenting safety cases to customers.