Work on unimportant problems

It seems obvious that some problems are important to solve and some aren't, as in, curing cancer is more important than delivering social gaming. Often, people lament the abundance of tech firms working on ultimately unimportant stuff, and advise to work on important problems and not just chase the money.

I guess I agree that some problems are ultimately more important than others. But I don't think it follows that working on the important ones is better.

Working on unimportant problems can create important side-effects. A whole lot of mission-critical, world-changing and even life-saving tech is a by-product of "unimportant" things – time-wasting infotainment products, or personal pet projects started without a grand noble cause.

For instance, GPU hardware was developed to run first-person shooters with increasingly fancier graphics. Today, it powers some of the largest high-performance computing clusters where "important" science is done.

Other types of processors powering HPC clusters weren't designed for HPC, either. Hardware originally designed for scientific computing is dead – Cray is the iconic example – and replaced by cheaper and more powerful microprocessors designed to run things like office software. Office software arguably solves no important problem: as Berglas convincingly argues, office automation results not in increased productivity, but in increased complexity of rules and regulations.

All popular programming languages and operating systems, without a single exception I can think of, began either as personal projects or commercial projects not aiming to solve any problem "important" by itself. People hacked on the stuff for pleasure (C, Unix, Linux, Python, Ruby, PHP), or to conquer the world of businessy/officy/enterprisey software (Windows, VB, Java, C#, ASP). One language more specifically designed for the implementation of important software is Ada – but most important programs are written in something else.

It can even seem that not much important computer hardware or software came out of institutions directly dealing with important problems. Much of the Internet protocols is one big exception, and arguably so is HTML – but probably not JavaScript.

And, certainly, it's the "unimportant" social companies that made publishing and coordination via Internet universally accessible. Myself, I'm not very fond of either Facebook, Twitter, etc. or the kind of political activity that's coordinated through these sites nowadays, but it's "important", without doubt – another important side-effect of unimportant time-wasting projects.

One might wonder how anything of importance can possibly come out of, say, FarmVille. I really don't know – however, I couldn't guess how anything of importance could come out of DOOM, and it did.

And then there's a reason why so much of the best tech comes out of the least "important" markets. These markets are big, and they're free. Important problems tend to imply a smallish scale, or heavy regulation, or both. So you can't finance the work, and/or can't get any work done anyway.

Consider the aerospace software market – there aren't many planes, but a whole lot of regulation. Philip Greenspun, a software entrepreneur, a flight instructor and an expert witness in both software-related and aviation-related lawsuits, had this to say about the Colgan 3407 disaster:

Who crashed Colgan 3407? Actually the autopilot did. ... The airplane had all of the information necessary to prevent this crash. The airspeed was available in digital form. The power setting was available in digital form. The status of the landing gear was available in digital form. ...

How come the autopilot software on this $27 million airplane wasn’t smart enough to fly basically sensible attitudes and airspeeds? Partly because FAA certification requirements make it prohibitively expensive to develop software or electronics that go into certified aircraft. It can literally cost $1 million to make a minor change. Sometimes the government protecting us from small risks exposes us to much bigger ones.

The same is happening in the automotive market, the healthcare market, etc. There's progress, of course, just nowhere near the progress in more frivolous areas – and much of the progress in "important" areas is a byproduct of progress in frivolous areas. As in, the best system for managing patients' records may well be Google Docs that doctors access from their iPads.

By the way, the importance of an issue correlates with the stupidity of rules, not just in technology, but in most things in life. The hoops you must jump through to get an "important" product out the door are not fundamentally different from airport security checks.

The airport security theater results in little added security. Likewise, the quality theater necessarily surrounding any life-saving technology results in little added quality. However, for much the same reasons, both are unavoidable. I've been working on automotive accident prevention systems for the last decade, and as time goes by, the regularly scheduled cavity searches are only getting worse.

So if you ask me – by all means, work on unimportant problems. They're often more fun to work on, and ultimately you never know how important they really are.