Suppose you have a sparse RAM API, something along the lines of:
- add_range(base, size)
- write_ram(base, bytes)
- read_ram(base, size)
People use this API for things like running a simulated CPU:
- define the accessible memory with add_range()
- pass the initial state to the simulator with write_ram()
- run the simulation, get the final state with read_ram()
Suppose this API becomes a runaway success, with a whopping 10 programmers using it (very little irony here, >95% of the APIs in this world are used exclusively by their designer). Then chances are that 9 of the 10 programmers are API users, and 1 of them is an API wrapper. Here's what they do.
API users
The first thing the first API user does is call you. "How do I use this sparse thing of yours?" You point him to the short tutorial with the sample code. He says "Uhmm. Errm...", which is userish for "Come on, I know you know that I'm lazy, and you know I know that docs lie. Come over here and type the code for me." And you insist that it's actually properly documented, but you will still come over, just because it's him, and you personally copy the sample code into a source file of his:
add_range(0x100000, 6) # input range add_range(0x200000, 6) # output range write_ram(0x100000, "abcdef") # run a program converting the input to uppercase print read_ram(0x200000, 6) # should print "ABCDEF"
It runs. You use the opportunity to point out how your documentation is better than what he's perhaps used to assume (though you totally understand his frustration with the state of documentation in this department, this company and this planet). Anyway, if he has any sort of problem or inconvenience with this thing, he can call you any time.
The next 8 API users copy your sample code themselves, some of them without you being aware that they use or even need this API. Congratulations! Your high personal quality standards and your user-centric approach have won you a near-monopoly position in the rapidly expanding local sparse RAM API market.
Then some time later you stumble upon the following code:
add_range(0x100000,256) add_range(0x200000,1024) add_range(0x300000,1024) ... add_range(0xb00000,128) ... add_range(0x2c00000,1024) ...
Waitaminnit.
You knew the API was a bit too low-level for the quite common case where you need to allocate a whole lot of objects, doesn't matter where. In that case, something like base=allocate_range(size) would be better than add_range(base,size) – that way users don't have to invent addresses they don't care about. But it wasn't immediately obvious how this should work (Nth call to allocate_range() appends a range to the last allocated address, but where should the first call to allocate_range() put things? What about mixing add_range() and allocate_range()? etc.)
So you figured you'd have add_range(), and then whoever needed to allocate lots of objects, doesn't matter where, could just write a 5-line allocate_range() function good enough for him, though not good enough for a public API.
But none of them did. Why? Isn't it trivial to write such a function? Isn't it ugly to hard-code arbitrary addresses? Doesn't it feel silly to invent arbitrary addresses? Isn't it actually hard to invent constant addresses when you put variable-sized data there, having to think about possible overlaps between ranges? Perhaps they don't understand what a sparse RAM is? Very unlikely, that, considering their education and experience.
Somehow, something makes it very easy for them to copy sample code, but very hard to stray from that sample code in any syntactically substantial way. To them, it isn't a sparse RAM you add ranges to. Rather, they think of it as a bunch of add_range() calls with hexadecimal parameters.
And add_range() with hex params they promptly will, just as it's done in the sample. And they'll complain about how this API is a bit awkward, with all these hex values and what-not.
API wrappers
If there's someone who can see right through syntax deep into semantics, it's the tenth user of your API, or more accurately, its first wrapper. The wrapper never actually uses an API directly in his "application code" as implied by the abbreviation, standing for "Application Programming Interface". Rather, he wraps it with another (massive) layer of code, and has his application code use that layer.
The wrapper first comes to talk to you, either being forced to use your API because everybody else already does, or because he doesn't like to touch something as low-level as "RAM" so if there's already some API above it he prefers to go through that.
In your conversation, or more accurately, his monologue, he points out some admittedly interesting, though hardly pressing issues:
- It's important to be able to trick a program using the sparse RAM API into allocating its data in specific address ranges, so that the resulting memory map is usable on certain hardware configurations and not just in simulations.
- In particular, it is important to be able to extract the memory map from the section headers of executables in the ELF and COFF format.
- Since add_range() calls are costly, and memory map formats such as the S-Record effectively specify a lot of small, adjacent ranges, there is a need for a layer joining many such ranges.
- An extensible API for the parsers of the various memory map formats is needed.
- ...
When you manage to terminate the monologuish conversation, he walks off to implement his sparse RAM API on top of yours. He calls it SParser (layer lovers, having to invent many names, frequently deteriorate into amateur copywriters).
When he's done (which is never; let's say "when he has something out there"), nobody uses SParser but him, though he markets it heavily. Users won't rely on the author who cares about The Right Thing but not about their problems. Other wrappers never use his extra layers because they write their own extra layers.
However, even with one person using it, SParser is your biggest headache in the sparse RAM department.
For example, your original implementation used a list of ranges you (slowly) scanned through to find the range containing a given address. Now you want to replace this with a page table, so that, given an address, you simply index into a page array with its high bits and either find a page with the data or report a bad address error.
But this precludes "shadowing", where you have overlapping segments, one hiding the other's data. You thought of that as a bug in the user code your original implementation didn't detect. The wrapper thought it was a feature, and SParser uses it all over to have data used at some point and then "hidden" later in the program.
So you can't deploy your new implementation, speeding up the code of innocent users, without breaking the code of this wrapper.
What to do
Add an allocate_range() API ASAP, update the tutorial, walk over to your users to help replace their hex constants with allocate_range() calls. Deploy the implementation with the page table, and send the complaining wrapper to complain upwards along the chain of command.
Why
Your users will switch to allocate_range() and be happy, more so when they get a speed-up from the switch to page tables. The wrapper, constituting the unhappy 10% of the stakeholders, will have no choice but fix his code.
Ivan drank half a bottle of vodka and woke up with a headache. Boris drank a full bottle of vodka and woke up with a headache. Why drink less?
Users are many, they follow a predictable path (copy sample code) and are easily satisfied (just make it convenient for them to follow that path). Wrappers are few, they never fail to surprise (you wouldn't guess what and especially why their layers do), and always fail to be satisfied (they never use APIs and always wrap them). Why worry about the few?
The only reason this point is worth discussing at all is that users offend programmers while wrappers sweet-talk them, thus obscuring the obvious. It is natural to feel outrage when you give someone an add_range() function and a silly sample with hex in it, and not only do they mindlessly multiply hex numbers in their code, but they blame you for the inconvenience of "your API with all the hex in it". It is equally natural to be flattered when someone spends time to discuss your work with you, at a level of true understanding ("sparse RAM") rather than superficial syntactic pattern matching ("add_range(hex)").
He who sees through this optical illusion will focus on the satisfaction of the happy many who couldn't care less, securing the option to ignore the miserable few who think too much.