Extreme Programming Explained

When I saw Kent Beck's "Extreme Programming Explained" in our office, I was shocked. I've already accepted the inevitable occasional purchasing of obscure C++ wisdom, exemplified by titles such as "Effective C++", "Exceptional C++", "Imperfect C++", "Modern C++ Design" and so on. Yikes. Oh well. As long as nobody uses the boost libraries in production code, they can entertain themselves by ordering whatever they want as far as I'm concerned.

But XP? A software development methodology? A programmer ordered a book on methodology? I have to find out who that was. Well, I found out and the shock became worse, because it was one of the cooler kids. Largely out of respect for that guy, I grabbed the book, took it home and read it. What follows is my book review. Spoiler: I say quite some positive things there.

XP and hire/fire dynamics

I didn't look for quotes on this subject, it's just that all of them caught my eye. Maybe it's because HR implications of things are inherently interesting, maybe it's because everything else in a "methodology" is life-threateningly boring, and maybe both. Anyway, the quotes are:

"Given the choice between an extremely skilled loner and a competent-but-social programmer, XP teams consistently choose the more social candidate."

Gotta love the "but". Well, what can I say about this approach? This approach would prevent the best pieces of software and hardware I've seen developed from happening. In all those cases, there was one or more "extremely skilled loners". More on that later, when we get to code ownership, open spaces, pair programming and the like.

For now, it's sufficient to say that a methodology preferring mediocre to "extremely skilled" and calling itself "Extreme Programming" is, um, interesting. That "competent-but-social" is a euphemism for "mediocre" is as clear to me as the fact that mediocrity, while related to lack of talent, is first and foremost a personal value.

"Here's a sad but repeated story: a development team begins applying XP, dramatically improves quality and productivity, but then is disbanded, its leaders fired and the rest of the team scattered. Why does this happen? … The team's improved performance shifted the constraint elsewhere in the organization. The new constraint (e.g. marketing, who can't decide what they want fast enough) doesn't like the spotlight."

I have a fairly developed imagination, especially when it comes to organizational dysfunctions. For example, I can imagine how a team delivering a good result without working overtime will be appreciated less than a team delivering a bad result through heroic efforts (this example is from the book, too; of course the unappreciated team is the one "practicing" XP).

Getting fired over productivity? Because marketing can't invent features fast enough? What??

Now, that certainly challenges one's imagination skills. But we aren't the kind of people to turn down a challenge, are we? OK, lez do it. Imagine. Imagine an organization so rotten, so disgusting, so peculiarly brain-damaged that it can fire people because they work too fast to invent new work for them. Hooray! We did it! I see it right in front of my closed eyes!

But wait, what's that? Are those unfortunate victims of corporate idiocy Extreme Programmers? The competent-but-social programmers? If they're so "social", can't they see what stinky a hole they've landed at? You don't get fired from such places; you quit. I mean, realizing that you're in a hopeless human quagmire is among the most basic social skills – even I have it.

I failed the challenge. I can't imagine this picture.

Brad Jensen, Senior VP, Sabre Airline Solutions: "If programmers won't pair or if they insist on owning code, have the courage to fire them. The rest of the team will bail you ought."

Aha. XP, the two-edged sward. Get fired for using XP or get fired for refusing to use it. Just ducky.

No, really. I don't want to be mean. I won't even pick on the VP's "courage to fire" wording. You know what – I'll even praise this wording. The employer depends on a strong programmer more than the programmer depends on them, and firing an evil, antisocial strong programmer is likely the most courageous act a manager can commit.

All I'm saying is, XP is basically this religious cult, as more quotes will show. This cult is spreading in your organization. They fire people. They easily fire exceedingly productive people, "skilled loners" that won't "pair" and insist on code ownership – nothing evil or antisocial. They fire people just for refusing to accept their most controversial "practices".

Could it be the real reason for the occasional extermination of productive XP teams, assuming Kent Beck is right and the disbanded XP teams he saw actually were productive? With all those "loners" I work with and the kind of "process" we use, I can easily imagine my reaction to a hypothetical XP epidemic at my workplace. Probably something along the lines of "Come and get me! Let's see who gets fired first."

XP "practices"

Why do I place the word "practice" in quotes? I dunno, do you know a better way to typographically mark stupidity? The word kinda reminds of "spiritual practices", the religion thing again. But primarily it reminds me that there's just too much money in this industry. I mean, I have a relative who makes and sells ceramics. Neither she nor her employees use any "best practices". There's not enough money in ceramics for "practices"; you have to work. No time to listen to people talking about work, hence no market for their talks and writings.

This term is annoying on so many levels that I could go on for ages. For example, the words "best practices" and "questionable practices" convey infinite amount of ignorant, arrogant idiocy. You could never pass an exam citing "best practices" – you are supposed to analyze the problem and prove your solution. But in the industry it's perfectly possible to get away with, and even get promoted by substituting analysis with rules of thumb, irrelevant to the case in point as they may be.

OK, enough said about the term "practices"; maybe it's just me. Let's talk about some practices.

Sit Together

XP likes open space. I don't like open space. I'm exceptionally good at ignoring external events when I concentrate, to the point where it takes me 10 seconds to understand what you're saying when you interrupt me in the middle of something. But there's a limit to this autism, and when lots of conversations I can relate to, intellectually and emotionally, happen around me, I can't work.

Many people have talked about the state of "flow" and how you don't want to interrupt someone when they're in it because they're extremely productive, and how interrupting someone all the time means they work at a fraction of their full speed. Myself, I "value" communication as this would be called in XP. I like to talk to people, and people like to talk to me. Lots of them. And much as like it, I like flow, too, and closing the door in my room is quite a Best Practice.

To be fair, with XP, you won't be able to concentrate behind a closed door, either, because of…

Pair Programming

We'll start with some quotes:

"Personal hygiene and health are important issues when pairing. Cover your mouth when you cough. Avoid strong colognes that might affect your partner."

"When programmers aren't emotionally mature enough to separate approval from arousal, working with a person of the opposite gender can bring up sexual feelings that are not in the best interest of the team."

"In Figure 6 the man has moved closer to the woman than is comfortable for her. Neither is making his or her best technical decision at this point."

"Programmers that won't pair." Sounds more like "programmers that won't couple."

"I like to program with someone new every couple of hours…"

OK, OK, this is neither emotionally mature nor in the best interest of the team. Enough with this sort of quotes.

XP measures time in "pair-hours", since all production code must be written in pairs. Since we programmers are good at math, it's easy to see that this halves your task force. Kent Beck claims that pairs are more than 2x more productive than 2 people working alone, so it's a net gain. Is that really so, even if each of the 2 people can close the door?

I worked in teams of size 2 or 3, with everybody going pretty much at full speed, and I can't imagine gaining back the 2x spent on sitting together. Maybe if you're doing something so extraordinarily boring and trivial that you can barely move without talking to someone about it.

No, really, I think XP is for writing huge piles of straightforward code. And here's my proof.

Shared Code

As we've seen, in XP, there's no code ownership unless you want to get fired. The code is owned by the team, and everybody can and is encouraged to change ("refactor") any piece of code, any time.

This concludes the triad of XP practices that I'll call "The Coupling Practices", both because of my emotionally immature amusement with the quotes above and because of the coupling between everything (it's one system owned by everyone, not separate modules with separate owners). The Coupling Practices are:

  • Open space
  • Work in pairs
  • No code ownership

Kent Beck raises just one objection to the shared code practice – people might act irresponsibly and make expedient changes. Well, paranoid attitude towards coworkers is not one of my many sins; if you trust someone to work for your company, then you trust them to act responsibly most of the time, don't you think? My objections are different.

First, the coupling. I'm a great believer in Conway's Law: Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations. I'm the great believer that I am simply because I've never seen this law fail in practice. Strong module boundaries mostly appear at social boundaries – code ownership and responsibility – or at technical boundaries (different programming languages, different processes, different machines, kernel space/user space).

If everybody owns all modules, and unless each module is a separate kernel module, they'll quite likely become one big hairy program. And I don't like coupling, at least not in this sense. Sure, XP encourages refactoring, which should take care of the problem; I just never saw anything beat Conway's Law. To me, it's like defeating the law of gravity with spiritual practices.

My second objection is that shared ownership can't work except for trivial code, and this is the proof of the "XP is for trivial stuff" claim I've made above. Let me start with an example.

I used to think that ASIC engineers, the kind that write hardware description code in Verilog or VHDL, are sociopaths. I mean, the ones I've met were very helpful and patient when it came to explaining things to my feeble programmer mind, but apparently they couldn't stand each other. If one of them even touched another's code, the other guy would go to pieces. As a programmer, I was used to the Shared Code practice (you don't need XP to have a happy tar pit of coders friendly patching each other's stuff). So these hardware types scared me off.

And then I began to understand what you were saying in Verilog. In hardware, you have a bunch of variables, and you have code that computes the next values of variables given their current values. Hardware is like a huge recursive function of its registers' state, with the reset logic being the base of the recursion. Now, it's trivial to understand how a variable is updated – it's spelled using the usual arithmetics and logic and if-then-else and stuff. But understanding how all these updates propagate, and what will happen in, say, 4 cycles, and how the pipelines work together is, I dunno, impossible.

So you see, when you have this mental model of how this monstrous state machine works, and someone makes a change to it, you tend to be upset, because it's hard to read someone else's change and update your mental model. And why are they so sure their change makes sense in the first place? They can't possibly understand everything there is to understand! You could say that it's all irrelevant since XP is about software, and this is hardware, but to me, the line is quite blurred. It's code in a programming language. You can run this code on a simulator or an FPGA without manufacturing anything, just by copying bits. It's software.

Now, obviously we have the other extreme – code so straightforward that you can read it from top to bottom and understand it completely. Say, a one-screen script shoveling through the file system, doing simple regexpy parsing and printing statistics. You know, Practical Extraction and Report. They even have a Language for it. The question is, where is most software – is it like a Verilog module or like a screenful of Perl? (BTW, my Verilog and Perl guru is the same person.)

Well, let's see. Low-level interrupt handling and scheduling code is definitely like Verilog – touch my code and I'll go to pieces. Compilers are also like Verilog, because you have complicated algorithms and you can't just read them and say "aha, I see why this is doing what it's supposed to be doing". There are lots of heuristics and lots of knowledge about interaction of passes and complicated data structures. Optimized image processing code is also like Verilog, because there are all the precision considerations and knowledge about the target optimizer and platform. Computer vision code is also like Verilog… Everything I currently deal with is like that.

I mean, code is not like descriptive prose you'd put in a local wiki. You can't read it, say "I get it!", update it, and make the author happy with the "refactoring". Code is more like poetry: change this line, and now the next line doesn't rhyme, or you've broken the rhythm, or you've put angry words into a happy poem, that sort of trouble. Which is one reason to like code ownership.

It's probably useful to have a second programmer with "read access" to important pieces of code, so that at least 2 people can help debug each piece. You can get there with code reviews, and without the Coupling Practices, which I can't imagine working except for code so straightforward that I doubt there's much of its kind.

XP and CMM

Brad Jensen, Senior VP, Sabre Airline Solutions: "The pure XP projects have very few defects… (Even) the (impure) XP projects have very competitive defect rates, one to two defects per thousand lines of code. The Bangalore SPIN, consisting of ten CMM level-five organizations, reports an average of 8 defects per thousand lines of code."

"Defects per LOC". Interesting metric. How do we improve it? First, we don't record "defects". What's a "defect"? Are 10 reported problems caused by one defect or several? Depends on the "root cause analysis", which is of course up to us coders. The second important thing to do is to write more code. That guy working on the data-driven layout rendering should be taken out and shot. We have editing macros for that. You could squeeze 50K LOC out of rendering if it was spelled as code like it should be.

Who is stupid enough to use a metric encouraging people to misattribute problem reports, increase code size, or simply quit the silly job? Why, it's the Capability Maturity Model, of course. Meet XP's competitor: the dreadful CMM.

I could have received a certificate telling that I was trained in CMM, but I couldn't take it and asked my manager to bail me out (and so he did, THANKS!!). It was stupid to the point of physical pain. The CMM instructor, making $160/hour, talked like this: "Blah-blah-blah-PRACTICE!! Blah-blah-blah-ESTABLISH-AND-MAINTAIN!! Blah-blah-blah-GOAL!!" – yelling at some random word, so you couldn't even fall asleep in the absence of any kind of rhythm your brain could learn to ignore. The sleep deprivation and the countless repetitions caused some of the garbage to be ingrained in my memory. Here's an excerpt for ya.

CMM has 5 levels. CMM level 1 is where you operate right now: it denotes the ability to ship something. CMM level 5, the one mentioned in the interview from Kent Beck's book, denotes complete paralysis. To do anything, you have to write or update so many documents, have so many meetings with "relevant stakeholders", and to perform so many pointless measurements of the defects/LOC kind, that it's much more productive to just go postal and at least remove a CMM auditor or two from the face of the Earth in the process.

Levels 2 to 4 indicate various intermediate stages where paralysis is spreading, but you can still ship. For example, level 2 is called "Managed Process". "Managed Process is distinguished by the degree to which the process is Managed". I'm not making this up. There's a book called Capability Maturity Model Integration, and this book, heavy enough to kill a human, is full of this sort of stuff. Reading it is impossible.

And this is why I think XP is a great thing. (See? I promised I'll say positive things; and I'm not done yet!) I've actually read Extreme Programming Explained. About 75% of it seemed meaningless, but I made it through. The CMMI book, on the other hand, is pretty much infeasible.

CMM has hundreds of practices. XP has a couple dozens. CMM has extremely costly certification process. XP doesn't. CMM forces you to write a zillion documents. XP forces you to write a zillion tests, documentation of features, and story cards. CMM is about lengthening the development cycle. XP is about shortening the development cycle.

CMM is inflicted on you by customers who believe the lies of the worthless bastards from the Software Engineering Institute that their "process" will make you ship quality software. XP attempts to establish itself as an alternative legitimate "process" in the realm of suspicious customers and costly vendors. XP tries to sell commonsense stuff like automated testing to the desperate programmers working on a single-customer product with scarce resources. XP also tries to sell itself to the customer, relieving the desperate vendor's programmers from the insane, intolerable overhead of level 5 CMM paralysis.

XP brings hope to dark, wet, stinky corners around the world. I sincerely think it's great. The permanent brain damage of that CMM training course makes me admire XP.

But then there are people who work for product companies and inflict XP upon themselves. It's like paying taxes the government doesn't ask you to pay. XP is an alternative to CMM. You need it if your customer or manager requires you to use a ready-made methodology, out of lack of trust. If you don't have that problem, make yourself your own "process". You are blessed with an opportunity to just work, and organize your work as you go and as you see fit. Why "practice" stuff when you can just work?

How many product companies use CMM, XP or any other ready-made process, and how many make their own process? How many successful companies borrow existing "methodologies" without being forced to do so? I don't know the numbers, but my bet is that most of them don't.

XP and religion

"EMBRACE CHANGE"

This quote is right from the book cover. "Extreme Programming Explained. EMBRACE CHANGE." Does it freak you out the way it freaks me out? Maybe it's because of the cultural gap created by my Russian origins? Nay, I know plenty of English slogans I can relate to. Say, "Trust a condom". Beats "Embrace change" hands-down. Changes come in two flavors, good and bad. Should I "embrace" both kinds?

"Even programmers can be whole people in the real world. XP is an opportunity to test yourself, to be yourself, to realize that maybe you've been fine all along and just hanging with the wrong crowd."

Is this a religious cult or what?

"The key to XP is integrity, acting in harmony with my true values… The past five years have been a journey of changing my actual values into those I wanted to hold."

"Journey". Talking about being good. Do you like hippies? I like hippies more than nazis. I like XP more than CMM. But IMO the hippie world view and general style is suboptimal.

"With XP, I work to become worthy of respect and offer respect to others. I'm content to do my best and strive always to improve. I hold values I'm proud of and act in harmony with those values."

"I have seen people applying XP bring renewed hope to their software development and their lives. You know enough to get started. I encourage you to start right now. Think about your values. Make conscious choices to live in harmony with them."

It's a religion, people.

I'm not strictly against religion. I even have vague respect towards religious people, those stating that their purpose in life is to be good and that they know how to do it. (I similarly respect the intentions and discipline of various development methodology followers.) Of course when a religious person behaves as an asshole, it's way more annoying than a secular asshole is, because the latter at least doesn't state that he's all about being good and living right. (I noticed that many people who love to talk about "proper process" commit the worst atrocities.)

Some religions or sects are awful, to the point of human sacrifices. (CMM.) Most modern mainstream religions are kinda nice though, and they promote good Values and Practices, like not killing, not stealing and helping each other. (XP promotes automated testing, short build, integration and release cycles.) Many religions have peculiar regulations when it comes to sex. (XP has The Coupling Practices.) This is when religious people start to get annoying, to the point of throwing stones at people who are improperly dressed, not to mention engaging in forbidden sexual relationships. (Refuse to "pair" and XP gets you fired.)

Interestingly enough, the majority of religious people feel much stronger about the peculiar regulations of their religion than the universally recognized atrocities, which are also forbidden by their religion. They never gather around prisons to throw stones at convicts going on vacation, they are more likely to do so near a gay parade. (The Senior VP from Kent Beck's interview didn't recommend firing over lack of tests, which all of us would recognize as a crime; he recommended firing over the violation of The Coupling Practices.)

And this is when my patience towards religious people evaporates. So there's a gay parade. Got a problem with it? Shove it! I'm not gay, but the idea of harassing people based on this sort of criteria is disgusting. (I think I can blend in very well in a team engaging in pair programming and collective ownership, but I know awesome programmers who can't work that way; just try to get them fired.)

But still, religion is not all bad. I think that there are religious communities which are less inflicted with drug abuse compared to your average secular community. (It's not unlikely that many XP teams have more tests and less bugs than "secular" teams using no "standard" process.) Few people have the courage and the patience needed to openly attack religion; in particular, holy scriptures tend to be long and obscure. (Skim through an XP or especially a CMM book and try to stay awake or find refutable claims.)

Young people who aren't cynical enough to realize that bullshit is free and hence everybody is going to bullshit you if only you let them tend to be an easy target for religions seeking expansion. (When I started working as a programmer, I was shocked how messy everything was, and cheerfully welcomed every "methodologist" about to create order out of the chaos. Today, even saying "Best Practice" near me is nor recommended.)

Summary

I treat XP as a religious lifestyle (true of any methodology). XP is not a bad religion. Its primary virtue is the threat it poses to the cannibalistic cults such as the CMM. While I don't want to be anywhere near XP churches, I think religion is an inevitable attribute of human existence, and as far as religions go, XP is not at all bad.

52 comments ↓

#1 levinv on 06.30.08 at 7:25 pm

Hi there,

As an acolyte of the XP religion, I never the less appreciated your blog posting. I do think that having some honest to goodness set practices is a good idea, and I ever believe that pair programming does add a lot of value in many ways. None the less, there is always a tension in any defined practice between discipline and dogma. On one side you wind up with a complete absence of standards; on the other hand you end up with people who start to look like a cult. The goal is to find a sane middle ground, but it's definitely easier said than done.

#2 jeremy on 06.30.08 at 9:59 pm

I agree with many of the things you've said, a few points:

* When you have a large company with a software product you need to maintain for decades then having code owners is a liability. You don't want loaner star programmers, which you can put up with in a startup, you want to write software that works and you can maintain.

* Pair programing may lower individual programmer output but it also lowers bug count. It's like unit testing – looks like it's slowing you down but actually speeds things up. Of course, for really good programmers there is no gain because they're so good at writing lots of good code with low bug counts but few programmers are like that.

* In the end the motivation here was to sell a book, for that you need enough wording to make up something people will buy.

* About the openspace thing – see PeopleWare…

I think, like any methodology you can just take what works for you out of it.

#3 levinv on 06.30.08 at 10:07 pm

I'd just like to add that the "coupling practices," to my mind anyway, are about helping the developers in the team evolve toward a *shared* mental model of the code. Combined with automated tests, this reduces the chances of introducing problems. To me the cost benefit is roughly: cost of introducing problems vs. cost to productivity of having only one person working on a piece of functionality over time without other people understanding what that person is doing, and of course the possible loss of that developer entirely.

Again, it's important to stress that professionalism and competence is a pre-requisite: You don't want people making willful or stupid changes. I definitely believe that raw competence is very loosely coupled with any methodology. In other words, some XP practices have, in my opinion, improved my programming skills, but having poor programmers follow XP religiously will not produce a particularly good result.

#4 Yossi Kreinin on 07.01.08 at 10:55 am

To levinv: the way to summarize my argument is that I think code should have a write lock :) As I said, it's OK and sometimes crucial to have other people who read the code in addition to the guy writing it. But for non-trivial there should be just one person making modifications, and a guy who reviewed the code and knows it shouldn't think that it's enough to make modifications sensibly. If the "write-locker" wishes to move on to something else, the write lock is released and one of the readers gets it. All I'm saying is that I don't like concurrent updates to the same code by several people, because of mental model synchronization problems. XP warns about the problem with long-living branches; why are they a problem? Because merges can make changes that were valid separately invalid in the merged version, even if automated merge passes. My worst bugs came from merges with myself – when changes done in different branches were finally brought together after a year or so. Several people making changes to the same tricky code is even worse.

#5 Yossi Kreinin on 07.01.08 at 11:06 am

to jeremy:

* regarding maintenance – to me "ownership" is the "write lock" from the comment above. If "ownership" means "read and write lock and intimidation of anybody trying to get near your code", then you get the problem you mention.

* I think pair programming lowers throughput dramatically as you can't enter the state of flow. Synchronizing a pair of brains all the time wastes a lot of effort. I like most to work in pairs on boring stuff which also can't be done very quickly because you have to shovel through a mess, and neither of you quite knows how to do each step at first glance. For stuff where you can get into a state of flow, or stuff where you need to concentrate deeply and look at ideas and pick some good one, I think you lose 10x speed on top of the 2x immediately lost due to pairing. It's hard to believe that the bug reduction coming from having two pairs of eyes gives you back the 20x. Perhaps you saw it work that way though.

* Wording – well, if you use wording appropriate for a self-improvement New Age booklet all over the place, I can guess who you're trying to sell that book to…

Regarding taking what works from methodologies – sure, the way secular laws or morality are not unlike religious laws, and you can see holy scriptures quoted at times in "secular wisdom"… I'm going to use the quote about overproduction being the greatest waste myself, and it's from the book (although itself some Toyota-related quote).

I just don't want to be anywhere near a religion or a methodology in its full glory.

#6 M@ on 07.01.08 at 12:53 pm

Very good writeup, and you are right on target. Try to get your mind around this one: I spent a year working for a large software shop that claimed to be both XP and CMMI. In your description, this would be like being a Christian Athiest.

I did as your article suggested anyone with a brain would do and quit.

#7 Yossi Kreinin on 07.01.08 at 1:10 pm

to M@: more like Christians sacrificing humans every Christmas or something. XP+CMMI=ZOMFG.

#8 Aristotle Pagaltzis on 07.01.08 at 2:50 pm

I just had to point out one thing, that “EMBRACE CHANGE” is not as nonsensical as you may think. And yes, it does indeed refer to both good and bad change. What it should say is “Change is inevitable – deal with it.” The slogan is about the core idea of “Agile” – inasmuch as there is anything worthwhile to those methodologies at all, it’s that Big Design Up Front does not and cannot work. Making up a complete design prior to any programming and following up with a phase in which that design is “merely implemented” is fantasy.

But you have to know that ahead of time for the slogan to make much sense at all.

PS.: I hate your guts for making me sign up for another account.

#9 Yossi Kreinin on 07.01.08 at 3:18 pm

Yeah, but do you really think this hatred of my inner workings compensates for my pride for finally getting your comment? :)

EMBRACE CHANGE doesn't seem to be about Big Design Up Front, although the book does mention BDUF, LDUF (Little…) and ENUF (no, not Enough…, something Else Design Up Front; I hoped there would also be a FUKOF, but no.)

EMBRACE CHANGE is about changing the way you think about software development, your life, the universe, and everything. Let's not play naive. The book is full of quotes about CHANGE, and most of them are about changing you, not changing your code. There's even a place where he says that people are more ready to accept CHANGE shortly after experiencing a failure (is that a note you'd expect a cult promoter to take or what?) Quotes about "us going about our business of changing the way people think about software".

You should CHANGE to become an XP believer. EMBRACE it already.

#10 del.icio.us bookmarks for June 30th, 2008 through July 1st, 2008 < Subject Code on 07.01.08 at 4:31 pm

[...] Extreme Programming Explained – [...]

#11 links for 2008-07-02 on 07.01.08 at 4:44 pm

[...] Extreme Programming Explained (tags: blog programming psychology management projects productivity software) [...]

#12 ProjectX Blog » Blog Archive » Xlinks - 2 / 7 / 2008 on 07.01.08 at 6:07 pm

[...] Extreme programming explained Added on 07/02/2008 at 01:40PM [...]

#13 KillJones on 07.01.08 at 9:13 pm

Loved this post. As a developer in an environment that is making steady progress toward becoming religious, you had me shouting, "preach on, brother, preach on!"

I often wonder if many of the advantages of XP are short-lived. For example, I have found it to be true that one produces fewer bugs when pair programming. This is largely because pairs tend to screw off less. But, if following a good TDD style, wouldn't bona fide hard-working developers be as productive? More productive?

I admire the collaborative development of XP. It is a truly good thing. All good things, though, run the risk of becoming bad ones (in ridiculous portions). If you don't believe me, go eat an entire bottle of vitamins and get back to me.

#14 levinv on 07.01.08 at 9:14 pm

Hi Yossi, I wonder if what you're referring to is a model along the lines of OSS development. As far as I can tell, in say the Linux kernel, there's a hierarchy of committers. If you want to supply a patch, you may try to get it checked into a branch somewhere, and eventually your check-in may percolate to a main committer who will then decide whether it is acceptable to go into the trunk for release in the "official version". I think this is major overkill for any kind of normal integrated development team though. In any kind of sensible software shop environment, I'd expect the following: a) don't just randomly check in a change to some piece of code you never actually have any reason to work with; b) if the code is complex, make sure you understand the underlying algorithms/principles being applied and talk to the guy who does most of the work on this code before making changes; c) include automated testing as part of the development process (they don't have to be XP "unit" tests. For example, in the case of concurrent code, you may have tests that run traces and check them for problems) .

#15 Entity on 07.02.08 at 4:30 am

Don't know about your opinion Yossi, but would like to hear it.

During my time overseas I had a great opportunity to talk to several business managers regard software development inside their organization. One of the most common thing i found, is that most organization would adopt curtain methodologies that suited their software development field. Though, they would not adopt everything, instead would use key aspects of the methodologie that suited them instead.

I knew each business owner personally, and talked with most of their staff well overseas. Smaller shops had the tendency to embrace software ownership and faster development turn around times. Larger development shops, would use different systems for different departments. Though each department seem to adopt or change a core design practices regardless of the management practices in place. Full enterprise shops would focus more on documentation, and less on code ownership, instead would be more reliant on commands coming from above.

One common thing they all had in common though. All of them were have trouble staying on-top of the work they had coming into the business, and keeping existing staff. So maybe it isn't really a problem with what methodologies you adopt or don't. But having a stable work force that knows your systems and keeping them.

#16 timander on 07.02.08 at 10:11 am

Thanks for the good read.

I am curious if you read the first edition or the second edition of the book, they seem to have quite a different attitude (the first one is a lot more cult-ish).

I agree that adopting practices without understanding the principles behind them is like being in a religious cult. I see XP as one implementation of practices to support Agile Principles.

Pairing programming is not for everyone all of the time.
Pairing is not necessarily going to reduce problems with your code base.

Pairing has been successfully for my team because it has accelerated learning, increased code clarity, and identified ways to simplify and test the code. It's like having continuous code review. It has also helped us minimize risk by distributing knowledge to everyone.

I also agree that someone who doesn't understand what the code is doing has no business changing something that is going to mess up everyone's mental map, but it might be appropriate for someone who needs a change to pair with someone who is familiar with the code base.

#17 Yossi Kreinin on 07.02.08 at 11:14 am

To timander: I read the second edition. Your remark makes me wonder what was in the first :)

To Entity: I didn't quite understand which opinion of mine you ask about :) Anyway, doing things mentioned in methodologies isn't using methodologies, it's using common sense. My problem with methodology is when instead of saying "I do whatever makes sense", you say "I do XP".

"Processes" aren't technology; no point or a good way to reuse them. Using an existing operating system instead of rolling your own makes sense. Using someone else's advices and wisdom instead of your own common sense in everyday life makes no sense unless you're a cult follower. Which is why sane companies rarely borrow process unless forced to do so for certification purposes.

#18 The Burgeoning Openly Owned Web » links for 2008-07-03 on 07.02.08 at 5:24 pm

[...] Extreme Programming Explained If ya need to bluff or just have an interest (tags: xp extreme programming) [...]

#19 joemama on 07.03.08 at 5:22 am

XP can't possibly work, never has, and never will, and anyone who suggests that any of its elements can ever work is a religious zealot. I don't for the life of me understand why people keep insisting that we "change," what we've been doing all along is working just fine, thank you very much.

Oddly, this article looks recent, even though the book is 4 years old, and the original is near ten years old. Obviously you've been keeping up with trends in software development, something every programmer as smart as you should be doing. Maybe you can rant about CASE tools next.

#20 DGentry on 07.03.08 at 5:54 am

I've never worked in an Extreme Programming shop. I suppose I'd be willing to try it, though I'd have this deep-seated worry that some day I'd be explaining to the police detective that my paired programmer had quibbled over the naming of loop variables one time too many…

#21 andy.edmonds.be › links for 2008-07-03 on 07.03.08 at 6:10 am

[...] Extreme Programming Explained If ya need to bluff or just have an interest (tags: xp extreme programming) [...]

#22 Yossi Kreinin on 07.03.08 at 1:10 pm

To DGentry: myself, I always liked working in pairs, and specifically I think that I'm very tolerable to style differences (so I'd gladly rename my loop vars to suit the emotional needs of the other guy). However, (1) that it's fun doesn't mean that it doesn't decrease productivity and (2) that it's fun to me doesn't mean it's even tolerable for other equally or more productive people and (3) 1 and 2 aren't compatible with the firing practices advocated by XP.

To joemama: being the lazy dimwit that I am, my policy is to let other people monitor trends to first see who tends to get interested. Then if the brilliant people like yourself love a trend, I of course immediately follow it without questions. If chaps in my league are interested, I take a closer look. As RTFAing would reveal, nobody seemed to care about XP around me for a long time, so I kinda didn't want to waste my time checking out a methodology.

#23 Weekly linkdump #133 - max - блог разработчиков on 07.03.08 at 9:35 pm

[...] шикарная статья с критикой XP методологии, Extreme Programming Explained. Там и CMM(I) досталось по ходу дела. Лучшая статья по [...]

#24 links for 2008-07-06 « The Adventures of Geekgirl on 07.05.08 at 8:46 pm

[...] Extreme Programming Explained (tags: programming xp development extreme agile blog methodology) [...]

#25 brynt on 08.21.08 at 9:56 am

You're spot on with the whole "I think XP is for writing huge piles of straightforward code.", but I don't think that's a bad thing. The majority of modern software development is about straightforward business code just applied slightly differently. Writing the actual code generally isn't that exciting, and the challenge becomes keeping people focussed on the task at hand. Pair programming keeps one from getting too distracted, and helps get the weaker members of the team up to speed. CMM and the rest of the heavily process driven models don't seem to consider "human boredom" as a primary factor at all.

#26 Yossi Kreinin on 08.21.08 at 11:09 am

I've wrote my share of businessy code. I'm not sure how straightforward it is on average. One aspect of this is that when you have to generate many printed reports, you can "straightforwardly" hand-code them all or you can have a data-driven printing engine, etc. Another aspect is that there are gnarly corners of behavior specification. And of course people are bored by different things, moreover, some care more about the process of coding and some care more about the result (working projects), and the latter kind is less prone to boredom.

But I dunno, maybe you're right and there's intrinsically more boring coding than any other kind (as opposed to "most code is boring because people do it wrong"). I can't judge the distribution of such things. What I can say is that a way to battle boredom shouldn't be called "Extreme Programming" or "Agile Methodology", and shouldn't be marketed as the best way of life for small teams trying to do interesting stuff quickly.

CMM surely considers boredom as a primary factor, albeit not in the way you mentioned – they use it as their number #1 weapon, boring their victims to the point of inability to resist.

#27 brynt on 08.22.08 at 8:34 am

Maybe "straightforward" not in the sense that the problems are easier than other kinds, but that it's a field where a problem you face will probably has already been worked on by a hundred other people who will have found answers to it and written it up on their website.

I think people get stuck when writing business programs more often through a sheer loss of motivation than for other reasons (not knowing what to do, not being "smart enough" etc). It witnesses that problem you pointed out in "cardinal programming jokes", where people claim they just don't know how to proceed but would suddenly change their tune if there was a million dollar carrot attached. By putting two people together XP tries to reduce the overlapping area in the Venn diagram of What Bores You. Even if both people are bored by the same thing, they're far less likely to give up/procrastinate/do the wrong thing if there's someone else sitting next to them.

The process oriented side gets it wrong when hearing "I just don't know how to proceed". It ignores motivation entirely, and believes that people just need to have their instructions spelt out in condescendingly simple deatils. Now there's two problems, unmotivated people and a ton of superflous process to follow, along with a nice papertrail so no one can claim they didn't know what to do.

Motivation is such a key part to programmer productivity in general, that while it could be named better, addressing the problems of a lack of motivation has to be a key part of any programming methodology.

#28 Yossi Kreinin on 08.22.08 at 9:04 am

I'll confess that I don't have the slightest idea how to motivate anybody. One of the very many reasons I don't want to manage anything. I used to think that most people would happily grind away if only nobody got in their way. I currently lean towards the opposite conclusion – most won't work unless there's a $1M carrot or a $100 chainsaw involved. Depressing.

And specifically with programming the thing is worse 'cause it's a mass profession and many people who can't stand it end up trying to do it for the money (that's why I became a programmer). I heard that on average, people quit programming after 5 years.

To which I reply, whatever. I ain't no manager, founder or any other sucker. Not My Problem.

That said, if you say that from your experience pair programming can actually put the fun back into the whole thing, I can only say "three cheers". As long as nobody tries to sink "skilled loners" in snake oil.

#29 programming by evolution? - Page 3 | keyongtech on 02.23.09 at 8:08 am

[...] to fix them. 2009-02-09 Today, i happened to run across a blog on the eXtreme Programing FUCK. http://www.yosefk.com/blog/extreme-p…explained.html Great article! Xah $B-t(B [...]

#30 Alexander Byndyu on 04.16.09 at 6:51 am

Thanks for post! Very interesting things for me.

#31 James T. Savidge on 07.17.09 at 9:43 am

Greetings,

You wrote: “Getting fired over productivity? Because marketing can’t invent features fast enough? What??”

Yes it sometimes happens, but nowadays they frequently call it being “laid off.”

If a productive team or a bad economy manages reduce the work back-log, a company will see the need for fewer resources and will try and save money.

Why “lay off” the productive programmers?

Level of productivity may not really be a factor in the considerations of who is to be let go. The “lower productivity” people are likely to be the ones who have been with the company longest, and their “institutional knowledge” might make it more expensive for the company to let them go that it is for the newer hires. The newer hires might be the higher productive ones, in part because of their lack of knowing how the company is supposed to do things.

Programmers that are so productive that not only does the company no longer need their fast output, but actually tells them to go away, often end up moving into contract and consulting work, where they expect the companies to tell them at some point: “Thank you. Now please go away.” :-)

Hopefully my use of quotes won’t be an overdone attempt to mark places where I see stupidity. ;-)

Thanks for the article.

James T. Savidge, Friday, July 17, 2009

#32 Yossi Kreinin on 07.18.09 at 6:21 am

Regarding the importance of backlog for job security: Iiiiiii don't know. There are two questions: (1) is the geometry of the real world inherently evil here and those with backlog-job-security really end up better off on average, and (2) should one strive to be in alignment with that geometry provided it is in fact evil as described in (1)? I say, let's assume the world is inherently good and try to waste less. Let's assume that organizations which fire for productivity are sick and we don't want to be there anyway. Darwin will judge who was right. Many an evil empire collapsed in front of me under the weight of its backlog.

Regarding newer hires being more productive on average while fired more frequently on average: again, it can also be the other way around. As you accumulate experience in the organization, you can use it to shield yourself from work, and you can use it to get more work done (by knowing who really needs what and who's really good at what and what will work out and what won't), and of course you can do a combination of those. The question is, what happens on average? Go figure.

I still say that the ability of marketeers to come up with features as a new bottleneck created by a productivity gain is the weirdest problem to cite.

#33 Jason on 07.22.09 at 1:47 pm

We're actually trying to take the best aspects of both ends of the spectrum and do what makes sense for us; we do some XP and we do some formalized CMM-3-like process.

It works for us. We're always looking to do as little as makes sense.

So, I am skeptical of the claims of both, but I believe both have positive ideas. As far as XP goes:
Others have written about XP, paraphrasing:

- If integrating is a good thing, integrate continuously.
- If testing is a good thing, write your tests first.
- If peer review is good, have your peer review as you write.
- If communication is good, sit together in an open environment.

So, the argument that XP seems to make is that there is never too much of a good thing. From my standpoint, that's not true in life, so why should it be true in software development?

#34 The Accelerated Weight Loss Program...That Never Fails! | 7Wins.eu on 08.25.09 at 11:39 pm

[...] The Weight Loss Diet and Exercise Guru » Blog Archive » Behind the neck press, good and bad carbs, cardio conundrumAOPA ASF Blog: Safety eJournal » Blog Archive » "A Random Act of God…"Timothy Dwight College | Yale University » Notes and NewsInternal Parasites: Protect Your Health — Hypnosis Network BlogTrading Outlook for CIT Group Inc. (CIT) | Hot Penny Stocks InvestmentsSpace Politics » Thoughts on the Augustine committee meetingU.S. Pharmacist » Blog Archive » Erectile Dysfunction: Beyond the Phosphodiesterase Inhibitors400 Bad Request [...]

#35 mcdonalds coupons on 09.13.09 at 6:32 pm

Thanks very much for this awesome entry.

#36 Matt Katz (mattkatz) 's status on Monday, 12-Oct-09 16:54:26 UTC - Identi.ca on 10.12.09 at 8:54 am

[...] http://www.yosefk.com/blog/extreme-programming-explained.html a few seconds ago from IdentiFox [...]

#37 Nathan on 10.12.09 at 9:11 am

Wow, that's a new kind of spam. Clever.

#38 Diego on 11.16.09 at 11:27 am

Great article. I'm in debt with the guy from slashdot that linked you.

Some time ago I tried to experiment pair-programming with one of my partners. The conclusion was:

Positives:
* Occasionally we learned something new from the other one
* Occasionally the non-typer did some good suggestions

Negatives
* Difficult if your job involves anything more than just coding (is there a programmer that just types all day long?)
* Requires the same level of knowledge in the subject domain , or this degenerates in a teaching session
* The detected errors are usually the most trivial ones (the ones a bit of testing should catch); but the mayor ones (usually in the design) are not discovered in this phase (again until actual testing)

Also, we embraced a limited version of test-driven development with good results. IMO the tests are some sort of extension to the specification, apart from their "problem-catcher" appearance. Nobody talks more about XP.

#39 Yossi Kreinin on 11.16.09 at 3:11 pm

Glad you liked it.

Regarding TDD the trademark/mantra – again, on one hand I've personally spent loads of time directly on testing in some contexts, on making tests easy for others in other contexts and on mandatory nightly testing in still other contexts and while it was one of the most mundane things I've worked on, it was (in perhaps a less than exciting way) among the most useful; but on the other hand I'm doing it as a secular activity and wouldn't like to join a testing religion.

#40 Eljay on 11.30.09 at 7:34 am

If some poor programmer is stuck in CMM hell (or alternatively Sigma Six paralysis hell, or the spawn-of-CMM hell known as TSP/PSP), and he can sell XP as an approved alternative in that CMM culture — and here's the trick — the programmer is not bound to by-the-book XP.

The XP programmer can do whatever he want, call it XP, and no one is the wiser. If someone calls the XP programmer on not following XP by-the-book, he can claim they are using an Agile variant of XP. And if continued to be pressed on that as being not-XP-enough, can claim they are using Lean XP hybrid, or Scrum XP hybrid, or Continuous Integration XP hybrid.

There's enough ammo there to fend of CMM (or Six Sigma, or TSP/PSP independent of CMM) indefinitely.

Now where did I put my Six Sigma black belt…

#41 Yossi Kreinin on 11.30.09 at 10:02 am

Perhaps this could work, although at places where CMM sells they like what they call identifiable work products (things one can print out to collect some dust), so you'd have to produce some of those matching whatever notion of XP you sell as an alternative to CMM. Not that you couldn't do it given enough persistence and willpower, just saying that fighting bureaucracy tends to generate bureaucracy as a side-effect similarly to the way fighting violence with government tends to generate some level of violence by the government as a side-effect. Overall I'm all for governments as well as grassroots counter-bureaucracy bureaucracy, in the few places when those are necessary.

#42 Ryan on 12.17.09 at 5:39 am

I had never heard of Conway's law until just now, and thank you for teaching me. It explains my company's product perfectly. Yesterday I had a thought, "Everything bad that happened today could have been solved by better business practices". Then I thought "wtf am I thinking. I must be old". Great article.

#43 Yossi Kreinin on 12.17.09 at 6:23 am

I think that Conway's law generally shapes things in good ways (not that there's much that can be done when/if it doesn't). If you compare the way things should be broken into subsystems based on articulated analysis and the way things tend to get broken into subsystems based on the evolution of responsibility of different people and teams, the second way turns out in hindsight to be neat in some unexpected ways, or more practical than the first way which turns out to have unexpected flaws or expected flaws of unexpectedly high practical importance.

#44 bfish.xaedalus.net » A tech blog recommendation on 03.25.10 at 2:14 pm

[...] On extreme programming (aka. XP, aka. Agile process) as a religious cult [...]

#45 Wayne Ariola on 09.28.10 at 4:32 pm

There is merit to the XP and the the latest craze over iterative approaches. This is a good article:

http://www.parasoft.com/alm

#46 Yossi Kreinin on 09.29.10 at 12:39 am

This isn't an "article", it's a page advertising a product. A programming-related product. This makes it a spam comment, whereas the phrasing passes the Turing test. Left hanging here for combining the two properties – never met human/AI-generated spam comments before, thought I'd keep the sample.

#47 Paul M. Parks on 12.30.11 at 8:25 am

Am I the only one who thinks that the slogan "Embrace Change" is a contradiction in terms? The word "embrace" connotes clinging; how does one cling to change? Or, maybe, the author was using the sense of adoption — I should adopt change?

It strikes me as being rather nonsensical. I think you're spot on in calling it religious.

#48 Yossi Kreinin on 12.31.11 at 3:15 am

@Paul: interesting site.

#49 bcat on 04.01.12 at 3:22 am

that would be the longest article I read in some time, good job :)

another parallel with religion – there are 2 types of people, the ones that do the right thing automatically when told "do what/how you want" and the ones that would do unspeakable/terrible things. Former would be confused by any unnecessary practices, later really need religion/methodology

#50 Vintharas on 07.24.12 at 12:26 pm

Omg that was a freaking hilarious read XDDDDD Awesome XDDDD

#51 Sigi on 10.03.12 at 10:39 am

Another *great* article, Yosef, and interesting discussion underneath (the best blogs are usually also the ones with the great comment flow, yours is no exception).

However, one link is still missing here:

http://programming-motherfucker.com/

#52 Martin on 10.30.13 at 10:46 pm

I might have an explanation of why/when XP genuinely works. If you have mediocre programmers that don't give a hoot about the work, having someone there will give you a sense of commitment, sense of comradery or a sense of being watched, whichever makes you plow ahead. When I was in high school I was recruited for pair-gym time. In this sense two bored programmers with some sense of having to work because (they're letting the other one down OR their laziness will be visible OR a sense of us against them OR something) will definitely outperform two bored programmers being bored separately. ….. you'll have to replace bored with the adjective that describes the actual situation. ….. and of course, if this theory applies, it would only cause an improvement for mildly un-engaged programmers that have some skills. I.e. not morons, assholes and not people who are skilled, productive and engaged, the former category is beyond the help of this religion, the latter doesn't need it.

You can graph the XQ (XP quotient, some kind of product of skills, motivation and ingredient X) and you'll get a nice bell shaped curve. Or so my theory predicts.

There is a formula for calculating the effect on productivity. But I won't write it here so I'm not killed by the armies of zealous followers. :-) LOL ETC

shrugs

Leave a Comment