Cycles, memory, fuel and parking

But to some extent, we already pay for memory use in cycles – the more memory you use, the more you tend to blow the caches (L1, L2, etc.) and damage your performance by large constants. When you start swapping out, that's just you blowing another cache.

If you believe that shaming developers about security bugs found in their code helps them avoid making more security bugs, maybe have a "reduce memory consumption day" like a "bug hunt day" and shame developers in whose modules memory consumption was reduced the most (without impacting performance, maintainability, etc.)

I don't know, but I'm reminded that in this paper's algorithms to auction both time and space, doing it for time is a *lot* simpler:

http://e-drexler.com/d/09/00/AgoricsPapers/agoricpapers/ie/ie0.html

(I say I don't know because our day-to-day problems are pretty far from the paper's setup.)

I could extend the metaphor here. Public transportation is like cloud services – you pay small for the small time you use the service. You don't pay for infrastructure (the car itself), only for what you use from it. You also can't really get the comfort of a personal transportation vehicle.

@gwer: it's true that we pay for memory use that way, but it's very erratic – you can preallocate a huge pool of objects and it doesn't affect your cache performance at all; I could go on, but the idea is, the price can depend on the memory footprint but it's far from a simple and/or continuous function.

@Z.T.: I guess I don't like to think of it as "shame", but it probably does work with security bugs because everybody knows their security bug is a very real problem they created. With memory, developers can reasonably think to themselves, "c'mon, I didn't care about memory consumption – nor should I!" – because memory consumption is not a real problem, at least until the moment when it is.

@Darius: interesting – I should read it.

@Stefan: I guess you could say that – especially where "public transportation" is actually a private service... If it's subsidized or the price is otherwise distorted, as it usually is, and if the decision which routes are available is made by some municipal body, then you start getting other kinds of special effects.

mmap()'d caches like Varnish implements might be one possible answer – as unused memory becomes more scarce, their performance drops because the probability of the object you want to retrieve being already paged in drops.

Any kind of cache that can lazily, dynamically adjust to the amount of memory left free ought to exhibit a similar gradually-changing performance characteristic.

Amazon S3 sure knows how to price memory...

Are you suggesting a memory aware programming culture which dynamically chooses the most effective algorithm to use for a task? For instance, a search task which dynamically chooses an algorithm using an O(1) hash lookup that optimizes Cycles vs. a O(log(n)) binary search that optimizes memory? The lookup would be preferred assuming the hash table can fit in memory.

One way to change the culture would be to make the developer + architect do 50 pushups for every byte they allocate on the heap.

"If a manager says, “reduce memory consumption”, do you say “Yes, sir!” – or do you say, “Are you serious? We have 100M available – but features X, Y and Z are not implemented and users want them!”"

As a programmer, I would say, "OMG, thank you!". It's managers who always push for more features, because they see pressure from users. Programmers want the time to clean things up, because it's where we work.

@alex: it's not that the manager says "reduce memory consumption, here's a timeout from other work" – just "reduce memory consumption" :) Another angle is that reducing memory consumption is not necessarily a clean-up, much like reducing runtime isn't – efficient code is frequently uglier. In this sense, pushing for efficiency is pushing for "features" both because it's a "feature" as far as a customer is concerned and because it "pushes for ugliness" as much as or more than any other feature bolted on under pressure.

Feature push and bad features are maybe the real problem here. I get surprised how much software goes from good to bloated and overcomplex because any old feature users ask for gets shoveled in without regard to the overall effect on usability.

Now if you need to reduce memory use for the same feature set then I guess you really need to come up with some draconian coding guidelines to avoid language features that lead to bloated code size, make all your own data structures, your own memory manager, etc., probably your own compiler is not a bad idea.

For a high throughput kind of program that's true, however for I/O bound tasks where latency is important this no longer works. A reduction in memory footprint directly translates into an ability to run more instances of a task in parallel, which is usually more valuable as raw performance is capped by I/O bandwidth anyway.

You can also see this in folding@home type projects (the less memory each task needs, the more you can cram on that card) and in realtime 3d graphics — especially consumer oriented versions of that (where you might well have to contend with tons of services/apps on an overly optimistic choice of hardware for the OS).

When I worked on console videogames, running out of memory was a vital concern (as you point out, when there's no swap, the price of memory suddenly goes to infinity). The way that we dealt with this was to maintain a memory budget for each project. Like a corporation deciding how much money it's going to spend on manufacturing, R&D, marketing, sales, back office, etc., we would assign memory by function. Code, stack, level data, sound effects, streaming music, 3d models, textures, and dynamic (garbage-collected) heap would each get their own budget. The budget was enforced by the memory management module, so that the pain of exceeding the budget was felt immediately by the department that was responsible for it. If you added another dozen photo-realistic textures, the game would refuse to load the level, and it would up to you to figure out how to compress or tile or share the textures, or simply do without. Of course, the requirements of the game might need the budgets to be revised, but this was a heavyweight process, and of course the department who budget was being proposed to be cut would push back, so it was nearly always easier to find a way to live within your means.

Good stuff. You need to be able to come up with a good budget and to maintain/change it over time for this to work, of course; some organizations are better at it than others. What I like about how processor cycles work is, if you have no budget, just a status quo, there's a pressure mounting gradually as things get worse, even if the org isn't great at managing a budget... Anyway, it's inspiring to hear that there exist places where the budget approach works well.

Have you read "The High Cost of Free Parking"? It seems relevant. The author rails against zoning that requires developers to supply huge amounts of free parking.

They also recommend that metered parking should be priced such that occupancy typically peaks at about 85%. Perhaps that can be mapped, somehow, to memory optimization on computers?

I don't think they consider the possibility of adjusting the parking price in real-time based on occupancy – the unpredictability of that method would be problematic.

Where I live they have laws taxing developers on too much underground parking in buildings... (Or so they say, I never checked the facts and heard at least one person claim it's an urban legend.)

85% sounds interesting, the question is what pricing mechanism to use.

Post a comment