Engineers vs managers: economics vs business
When chased by a bear, engineers want to run faster than the bear, managers want to run faster than you. This is known as
"the best vs the good enough", and is a very common theme.
For instance, company A releases a good enough technology, company B releases the best technology on the market. B fails and
A succeeds, because A releases earlier, or because A's technology is more compatible with the status quo, etc. Engineers will
commonly feel sympathy for B, managers will applaud the shrewdness of A.
It's a common story and an interesting angle, but the "best vs good enough" formulation misses something. It sounds as if
there's a road towards "the best" β towards the 100%. Engineers want to keep going until they actually reach 100%. And managers
force them to quit at 70%:
There comes a time in the life of every project where the
right thing to do is shoot the engineers and ship the fucker.
However, frequently the road towards "the best" looks completely different from the road to "the good enough" from the very
beginning. The different goals of engineers and managers make their thinking work in different directions. A simple example will
illustrate this difference.
Suppose there's a bunch of computers where people can run stuff. Some system is needed to decide who runs what, when and
where. What to do?
- An engineer will want to keep as many computers occupied at every moment as possible β otherwise they're wasted.
- A manager will want to give each team as few computers as absolutely necessary β otherwise they're wasted.
These directions aren't just opposite β "as many as possible" vs "as few as necessary". They focus on different things. The
engineer imagines idle machines longing for work, and he wants to feed them with tasks. The manager thinks of irate
users longing for machines, and he wants to feed them with enough machines to be quiet. Their definitions of "success" are
barely related, as are their definitions of "waste".
An engineer's solution is to have everybody submit their jobs to a queue managed by a central server. The Condor software is an example implementation; there are many others, with
many subtle issues and differences β or you can roll your own. The upshot is that once a machine becomes idle, it can
immediately yank a next job from the server's queue. As long as there's anything left to do, no machine is idle. This is the
best possible situation.
A manager's solution is to give the ASIC team 12 servers: asic01, asic02, ... asic12. QA gets 20 machines, qa01 ... qa20, and
so on. If you want to work on a machine, you ssh to it and you work. If you're an ASIC engineer and you want to use a QA
machine, you can't log in. If users fight over their team's machines, they can go to their manager. If their manager decides the
team needs more machines, he goes to upper management. Good enough.
The "good enough" is not 70% of "the best" β it's not even in the same direction. In fact, it's more like -20%: once the
"good enough" solution is deployed, the road towards "the best" gets harder. You restrict access to machines, and you
get people used to the ssh session interface, which "the best" solution will not provide.
Which solution is actually better? Tough question.
- The manager's solution requires no programming or installation, and trivial administration.
- The manager's solution requires no changes of habits (ssh is the standard).
- The engineer's solution yields ~100% utilization, the manager's 50%, 30%, or 10%, depending.
- The manager's solution will cause less headache to the managers, on average. Once in a while, you buy a team some more
machines, and they shut up. The engineer's solution requires to set priorities at times of overload, and people will constantly
argue about priority with managers.
- The engineer's solution doesn't run things on machines that are already busy anyway. Users armed with ssh will tend to do
this, possibly with horrible slowdowns due to swapping, etc.
- The engineer's solution can provide the lowest latencies.
- The engineer's solution makes it trivial to utilize new machines β no need to set up sessions, just submit jobs as
previously. This is good β and bad: people will demand new machines instead of reducing the load.
So there are many conflicting considerations. Their importance varies between cases, and is very hard to measure.
"The best" solution is not necessarily the best β rather, it's what the mindset of looking for the best yields. Likewise, the
"good enough" solution is not necessarily good enough β it's just what you come up with when you look for a good enough
approach.
Obviously, portraying "engineers" and "managers" this way is a gross oversimplification, and most real people can look at
things from both angles. I like this oversimplification for two reasons:
- It does help to understand and predict many common arguments and reactions.
- A person's perspective frequently does coincide with the title: engineers aim at the best, managers look for the good
enough. Just the title is thus a good basis for prediction.
How does title affect judgement? There are arguments to the effect of "aiming at the good enough is wiser, and managers are
in a better position to achieve wisdom", or vice versa. However, I don't believe either viewpoint is more correct than the
other. Rather, they are biases. People in different positions acquire different biases, because they have different incentives
and constraints.
The difference in incentives is that engineers are rewarded for success, while managers are rewarded for non-failure.
An exceptional engineer is unique and valuable, like a great chef β being an OK engineer is more like cooking for McDonald's.
An engineer needs unusual success to be noticed. One reason such success is achievable is because he works within his area of
expertise, on things that he understands. There's rarely a formal quality metric, but his deep understanding creates in him a
sense of beauty that serves as his metric.
A manager is fine as long as the project doesn't fail. Most projects fail β if yours didn't, it's quite an achievement. One
reason they fail is that there are a lot of ways to fail. Forget just one item in a lengthy checklist, and it won't matter how
well the other 99 items are handled. People may not buy a car because there's no convenient place for a resting arm. A manager
looks after a whole lot of items, usually without being able to understand most of them in any depth. He's inclined to look for
simple pass/fail criteria.
If your success function is a continuous metric, you'll aim at the best. If it's a long list of Boolean values β V or X β
you'll want "good enough" (V) at every entry. "The best" is just a costlier way to get a V, at a higher risk to end up with an
X. The manager's outlook leads to binarization.
Another difference is that engineers and managers control different things β "One man's constant is another man's variable". In our example, the engineer
doesn't decide how many machines to purchase or how to allocate them. If the number of machines is constant, what remains is to
make the most of them. The manager, on the other hand, doesn't directly control utilization, and frequently doesn't know how to
improve it. If utilization is constant, what remains is to properly ration it.
More generally, engineers tend optimize within constraints set by management, in part because they can't change those
constraints. As to managers, they tend to settle for "good enough", in part because they can't optimize.
Which biases lead to more sensible solutions depends on the situation. However, there's a subtle reason to like the
engineer's devotion to excellence, despite the manager's pressure for mediocrity. It's precisely when the manager is right in
that excellence will not contribute to sales when the engineer contributes the most to users.
If an engineer's job is done so poorly as to make the product non-marketable, the product will fail and won't be used. When
the quality along all dimensions passes the adoption threshold, every improvement along any dimension is
effectively a gift to the users β who'd be using the product anyway. This is false where improvements contribute significantly
to popularity, but true where they don't.
The improvements are inconsequential for business, but consequential economically. Economically, anything
that helps people is good. For business, anything that creates profit is good. Often these definitions of "good" coincide, but
sometimes they don't. In these cases, people who aim at excellence for excellence's sake help bridge the gap.
The manager's checklist approach β the business angle β ensures that the product is marketable, which is great because
otherwise it won't get used. But the engineer's urge to optimize is closer to the really important goal: making the best of what
we've got; this is the economics angle. So while it can lead to problems just like any other bias, it's likeable that way.
"Engineers will commonly feel sympathy for A, managers will applaud
the shrewdness of B." β you clearly meant the opposite
"asic01..asic12, qa01-qa20" β sometimes, the manager's solution is
implemented on top of the engineer's solution (the user visible machines
are actually VMs running on top of a cluster of real servers) and
sometimes the engineer's solution is implemented on top of the manager's
solution (each department's machine runs a distributed work client the
queue for which is accessible to other departments). This makes things
more complicated, both socially and technically.
"One reason they fail is that there are a lot of ways to fail." β
"Happy families are all alike; every unhappy family is unhappy in its
own way."
"engineers tend optimize within constraints set by management, in
part because they canβt change those constraints" β necessity is the
mother of invention. See Unix and C on the PDP-6. All art forms require
constraints.
"When the quality along all dimensions passes the adoption threshold,
every improvement along any dimension is effectively a gift to the users
β whoβd be using the product anyway." β companies frequently won't spend
money on improving their product if they already have the best in the
market. They will even buy patents for a better product and sit on them
instead of building it, because that's the most profitable thing they
can do. A corporation's job is not to build the best product β it's to
make money. Only fear of a competitor who does not agree to hold back
and make money will lead a corporation to invest heavily in R&D.
Markets naturally devolve into oligopolies that collude (with each other
and government) to maximize profit and destroy newcomers who wish the
destabilize the status quo (through technology or simply by reducing
margins).
"Economically, anything that helps people is good." β what is this
definition of "economy"?
The best products possible, disregarding profit, can be made as
public works, using public funds or donations by artisans chosen in a
contest. Remove profit motive and leave only reputation on the line to
ensure quality work. Unfortunately I don't know how to avoid corruption
in the panning committee.
Regarding A and B β thanks, fixed, I probably wrote it that way
because it sounded naturally with first A, then B. It's part of the
surprisingly hard problem of assigning letters such as A and B to one's
imaginary entities...
Regarding one solution on top of the other β yeah, quite so.
Regarding companies not spending money on R&D β well, in that
case I meant to whoever worked on features that are non-essential,
sales-wise, before R&D spending was completely stopped. The extent
to which desirable improvements won't happen for business reasons is
debatable, and my guess would be much more optimistic than yours (I'd
estimate a smaller extent of the problem), but almost everyone agrees
it's non-zero.
Regarding the definition of economics β well, one definition of the
subject of economics is "allocation of scarce resources to alternative
uses", and "helping people" is a vague way to describe the stated goal
of a "good" allocation. To define this goal, you need a moral framework;
in the Anglo-Saxon world, two popular frameworks are utilitarianism (the
greatest good for the greatest number) and Rawlsianism (favor the
worst-off). Utilitarianism sort of leads to "pro-growth"/"free-market"
policies and Rawlsianism sort of leads to "socialist" policies;
"anything that helps people" sort of captures the common in their stated
goals.
Regarding removing the profit motive β corruption aside, earning
reputation is not fundamentally different from earning money, in the
sense that there are still cases where something can be done that helps
but will not significantly affect your reputation, so the public
interest and the private interest don't coincide; in that way it's
qualitatively a similar situation. Whether it's quantitatively better I
don't know.
I think this is underestimating what engineers do.
Most try to resolve the tension between economics and technical
perfection.
Managers don't ensure their products are marketable. Most will lie
directly to the rest of management about what they have achieved while
subverting any real value generation at all by engineers. When you
consider this propensity to lie and deceive, good enough is really just
good enough to fool all the other managers, many of whom have no
technical basis to even understand the products being developed.
In your example, the engineer's solution is the best in the long
term. The 'good enough with lies' approach eventually leads to collapsed
companies and failed products. However, by the time they collapse, the
managers will be long gone.
Additionally it is unfair to compare the manager's solution to the
engineers simply on the basis of the manager having unlimited ability to
add more resources. This is just an example of 'might makes right'.
At the end of the day, managers can't ignore basic economics. At some
point all the scheming, lying, and shortcutting always causes companies
or products to collapse just as a gambler eventually goes bankrupt.
@anon: well, it equally underestimates what managers do, since they
try to resolve this same tension, too (though I'd put it in slightly
different terms); it's just the bare bones bias of the respective
positions, at least the way I see it.
@noone: I don't think it's that bad, really.
Regarding "best solution for the long term" β as was mentioned above,
there are also hybrid solutions; one reason is that both solutions I
mentioned have their drawbacks in their straightforward forms.
Regarding "might makes right" β the manager in my example is trying
to buy less machines just as much as the engineer, the question is who
succeeds more in that and in helping people getting work done.
Technical debt gets solved how?
Thanks, nice.
Question! Who builds this companyβs products? You do. Engineers do.
Not managers. They should be answering to you, not you to them! Who else
has an idea like this manβs Coke machine? All right! Tell me. Better
yet... can anyone show me?
You forgot one thing: a manager's 70% solution will come back to bite
him in the ass one year later when he wants to quickly
extend/scale/improve the product and fails.
There's quite some anti-manager bias in the comments. Something to do
with the 99% being 99x more people than the 1%?
Trite article. As an example: "Suppose thereβs a bunch of computers
where people can run stuff. ". Where? At your site, the customers,
Amazon?
Engineers solve problems; financial as well as technical. Managers
should help them do this.
I don't exactly see this fitting Apple's model... everything I've
read indicates that they strive for excellent, from the management's
acceptance criteria down to the engineer... exceptions to this goal may
very well have meant that their products were NOT the success that they
became.
@Scott
I think you might be passing up some important information about
Apple. While they may appear to be polished on the outside, really look
at their products and you will find they almost always launch a 70%
version first. (i.e. no front camera, ipad no camera, no cut and paste,
no multitasking). They always come out with a version 2 or 3 or 4 to
improve on the last 30% but the market continues to move and they stay
behind. It is the simplicity offered by not doing the full 100% that
gains them users. Very few users need the full 100% and 70% seems to
work just fine.
Apple seems to fit nicely into this 70% structure.
Managers may not understand, but it's their job to get good,
knowledgeable people to advise them. Think Captain Kirk: he wasn't an
engineer, he wasn't a navigator, he wasn't a radio operator. He knew
enough of each of those to be passable, but there were clearly parts of
each that were beyond his expertise. However, he could see the
big-picture elephant, not just little parts of the elephant like the
blind men in the poem.
That said, I know from experience that having a manager who does
understand the product can be a blessing. I've worked under a couple
such managers, and also under managers who were ignorant asses. One such
ignorant ass told his subordinates that he had promised delivery of QA
tests within two weeksβwhich would have taken the staff more man-hours
to develop than they would have if they worked 24/7. They went to their
boss's boss, the division VP, and explained the reality of the
situation. The ignorant ass didn't last long after that, but it didn't
matter. The old COBOL code just never worked right on Windows, and the
company was bought out and ransacked a year later.
Oddly, the same thing happened where I worked with an excellent
manager, but that was a strategy to get the product some marketing that
it desperately needed.
Seriously? You worked on COBOL code that shipped on Windows?
COBOL for Windows? Hell yes. Haven't you heard of COBOL.NET? http://www.netcobol.com/cobol-compiler-for-net/
I've heard of it, but never met anyone who worked with it.
Well said!
this page is really good!
this one one made me realize the differences between managers and
engineers..!
this one is funny!
You've nver met anyone who worked with COBOL on Windows? In my second
"real job", in 1997, I worked for Keane on a Y2K remediation project.
For sniffing out date-handling logic, we had a thing called the "Keane
Assessment Tool", which was basically fgrep with a GUI, written in Micro
Focus COBOL and only runnable on Windows. I quickly hacked together a
replacement in Perl that I could run on the Suns, which let me do more
code scanning in an hour than I could have done with KAT in a week.
Unfortunately, my people skills (and UI design skills, I suppose) were
not up to the challenge of getting other people to use it, particularly
since that would probably mean billing the client for less hours.
How billing for hours works is something that generally escapes me,
because clearly the client is asking for trouble by encouraging you to
work for more hours than needed. I think one theme there is that clients
know that something should take at most N hours and will get annoyed if
you spend more time on it; and, if you're done faster, and it's not a
one-off project, you're more likely to get the next one. But I'm really
not sure how this type of arrangement should be made to work.
Wow! Really really good article. The "good enough" vs "best" had me.
I appreciate your approach of "gross oversimplification"
Thanks
I am a undergrad industrial engineer and what I've heard from most
professors in both business and engineering is that as an IE my job is
to present my employer with the best possible alternative wich is often
the most expensive route and let those on the business side make
adjustments that are then sent back to me to sign off on based upon
project feasability.
I am a undergrad industrial engineer and what I've heard from most
professors in both business and engineering is that my job is to present
my employer with the best possible alternative wich is often the most
expensive route and let those on the business side make adjustments that
are then sent back to me to sign off on based upon project
feasability.
Well, you're presenting it from the angle of "whose duty it is to do
what", which deserves a separate discussion and I guess depends a lot on
the organization. I was mostly looking at how people tend to think, not
how they set the boundaries of their responsibilities; here both
engineers and managers can see their responsibility as very broad or
very narrow, and things will accordingly work very differently.
I have the opinion that in fact, Managers, when attacked by a Bear,
tend to cover you in Honey and speak in bear- ish to the bear to try to
convince him /her how tasty and delicious you are...
Why such hatred towards managers, fellas? If it wouldn't be for them,
who would market your products? What would you do with your 'the best '
stuff? No doubt engineers are amazing, but then, so are managers. If
they follow the good enough strategy, please understand it's what you
call survival policy. How to make the best out of the least possible
resources. That's called intelligent business. That doesn't imply they
cut down on what you need, they're just trying thto avoid what you can
do without.
Thanks, you have made me realize that the past three years of my
management degree has been a complete waste of time. I will now start
again on an engineering degree as I will never be recognized as a
manager.
As an Engineering Manager I recognize the natural conflict that this
brings. Being an Engineer before a Manager obviously tilts me in the
direction of 'Fit for purpose' first and foremost. But then when the
customer wants it yesterday, a compromise must be reached (i.e pulling
back from the 100%). I call this 'Sprinkling some magic'
Good one....
Post a comment