Terse Systems

New 5MOF presentation, wherein I try to talk about why web applications are complicated. In five minutes.

Ty put up a video of me presenting Happiness @ Noisebridge.

It has kittens.

I've been thinking lately about Agile.  Again.

The first thing I've been thinking about is the people who say "You're doing Agile Wrong."

There's always been a dichotomy for me between the theory of Agile, and the practice.  It's a common problem with any dream; it's always cleaner, brighter, simpler, better than the reality.  Reality is messy.  It is imprecise.  It is never seen directly, always filtered through recollections to make each participant the protagonist of their own play.  

If you try pair programming, then you're going to find that "you should never pair program 100% of the time."  Or that "you should only pair program between people with equal skill sets."  Or that "you should practice pair programming ping pong".  There will always be a special case. There will always be something that works for you that doesn't work for someone else.   There will always be something that doesn't work for you that works for someone else.

Not only do we do Agile "wrong", but we will always do Agile "wrong".  We won't ever do anything "right" -- we will do imperfect jobs, come home to imperfect relationships, have imperfect children and live imperfect lives.   This is what happens when you measure yourself against an ideal.

But why believe in Agile then, if the only way you can do it is wrong?  Something that came to mind about that statement.  

If you're trying to do Agile, and it's not working for you... then you're doing it wrong.

Another way of phrasing that statement is that Agile is Doing It Right.  

In fact, almost by definition, Agile is Doing It Right.

"Agile teams produce a continuous stream of value, at a sustainable pace, while adapting to the changing needs of business." -- Elizabeth Hendrickson.

"Agile development uses feedback to make constant adjustments in a highly collaborative environment." -- Practices of an Agile Developer.

"Agile has no definition. [...] There's no standards board, there's no test, there's no approved workbook, there's no checklist.  [...] It's based on three things: 1) principles not practices, 2) attention to people, and 3) always be adapting." -- Daniel Markham.

Three definitions of Agile.  Nothing about practices, or even methodology.  What they agree on is a feedback cycle that can respond to changing input and produce useful output.  

It's Dorner's model of problem solving.  Or Deming's PDCA cycle.  Or the Military's OODA cycle.   Or the Scientific Method.  Or Kaizen.  It's continous process improvement, in all its forms.

If you're following a "best practice" and that "best practice" isn't working for you, then it's not a case of "You're Doing Agile Wrong."  You're doing something that isn't providing a benefit for you.   By following that "best practice", you're not doing Agile at all. Agile is the ability to plan something new, throw out something old, and challenge preconceived beliefs.

That's the core of Agile for me: it's not about Wrong or Right.  It's the idea of saying "We can do better."  And then doing it.

Another talk, this time on The Logic of Failure: Recognizing and Avoiding Error in Complex Situations, a book by Dietrich Dorner.

The book’s been a favorite of mine for years, not just for the set up, but for the detailed, unsparing look it provides on how human beings fail to get things right.  Too often in psychology, there’s an emphasis on either seeing how people feel about a situation, or how well or how poorly they perform at a given task.  Dorner goes further, and tries to understand not just how, but why they fail.

The Slides

The Setup

The setup was simple.  Dorner set up a computer simulation of an African village called Tanaland.  This book was written in 1990, and so Sim City was not widely known, but it’s the same concept.  The players were given dictatorial powers, given the goal to “improve the wellbeing of the people” and had six opportunities over 10 years to review (and possibly change) their policies.

The Experiment

Given the tools the players had at hand, they went to improving what they could.  They improved the food supply (using artifical fertilizer) and increased medical care.  There were more children and fewer deaths, and lif expectancy was higher.  For the first three sessions, everything went well.  But unknown to the players, they’d set up an unsustainable situation.

Famine typically broke out in the 88th month.  The agarian population dropped dramatically, below what they had been initially.  Sheep, goats and cows died off in their herds, and the land was left barren by the end.  Given a free hand, most players engineered a wasteland.

One player, by the end of the simulation, had a stable population and had significantly better quality of life for the villagers.  Failure was the rule, but somehow he had found an exception.

The Breakdown

The litany of possible errors was a long one, and so immediately recognisable that it's hard to suppress a wince of empathy on reading.

The players who did badly tended not to ask "why" things happened.  They tended to jump from one subject to another, switching aimlessly, without focus.  They proposed hypotheses without testing them.  If they did test their hypotheses, they did so on an adhoc basis, testing success cases without testing possible failure cases.  In some cases they had tunnel vision: focussing on irrelevancies at the expence of the larger picture.  In other cases, they attempted to "delegate" intractable problems to the villagers themselves or refused to deal with the issue at all.  Finally, and most tellingly, most players dealt with the problems that they saw "on the spot" without thinking of the larger, longer term problems that they were setting up with that immediate short term solution.

These results were not a surprise.  They were just what Dorner's team was looking for.  Where many scientists would have looked at the successes and determined the optimal "working strategy" -- Dorner was just as interested in the range of failures in the experiment.   Dorner's team had specifically designed the simulation so that most people would fail at it, precisely aiming at the weak points of human decision making.  

Not all players failed in the same way.  Even amongst the players who failed the same way, many players had different reasons for their particular mode of failure.  And yet, there were strong commonalities among the failing players, both in their reactions to incipient failure, and in their attempts at recovery.

The Reasons

The reason why most people failed was that they did not understand the nature of Tanaland.  Despite being a simulation, Tanaland was no game, and Dorner's team programmed in as accurate a simulation of an African village as the hardware would allow.  The watertable under the village had a limited amount of water available.  The population grew at an exponential rate given the available food and healthcare.  Even the topsoil was modelled accurately, so that overgrazing caused by massive herds would erode the topsoil over time.  All of this data was available to the players -- had they thought to look.  But most players didn't.  The experiment ended in three predictable failure modes: either the cattle starved and died, or the groundwater was exhausted, or the population exceeded the available food.  Far from being a bundle of independent subsystems, all of Tanaland was deeply intertwingled.

The Weaknesses

The deeper reason why Tanaland was so successful at bamboozling players is partly due to the incredible success of the human brain's pattern recognition system.  Human beings are capable of driving in heavy traffic, understanding language, and recognizing patterns in almost random data, feats far beyond most computers.  But there are some problems which defeat human intuition.  

Linear extrapolation. Human beings have a tendency to assume change itself is static.  Even when shown exponential growth, we're not good at internalizing that knowledge.  This may be why calculus is so hard for many people, because we don't think about the rate of growth itself growing.

Delayed Feedback. Human beings tend to assume that an action will yield a response immediately, or not at all.  This is the way that we interact with the world on a daily basis, and we can become very confused when there's a significant delay in the system's response.  Even when we recognize intellectually that a change is "in the pipeline", we may struggle against the instinct to do more and oversteer rather than correctly "sitting on our hands."

Contradicting goals. Part of the failure of players was inherent in the vague goals that they had.  What, exactly does "improve the wellbeing of the people" really mean?  Does it mean providing the best quality of life to all the villagers?  Growing the village as a whole to be more prosperous?  In many cases, the top level goal ended up being broken down into goals that conflicted with each other.  In other cases, players tried to find concrete problems to fix.  One player, deciding that the village needed irrigation, set out building an irrigation system and quickly became fixated on that one problem, becoming "addicted" to his experience of flow.  In such cases,  players were unable to clearly form goals at all.

Priorities. Even when the players had clearly defined goals, they had another problem to contend with: they would be stymied by cases where actions which furthered one goal would thwart another.  The complex interdependencies in the system did not allow for a full optimization of every variable, and players would either flail uselessly or be paralysed by their inability to cover every base.

Information overload. In many cases, the players used too little information to know how to make the best decisions.  However, some players had the opposite problem; given access to all data of a complex system, they tried to see the entire system at once.  These players found themselves paralysed by complexity, and unable to interpret the results of the data.  Interestingly, the problem the players had was not that they did not see the correct chart.  They were literally unable to recognize the charts and the changes in data as relevant -- having looked at all the data available, their abilities to see a pattern was exhausted well before they stumbled on the correct chart.

Reductive Hypotheses. By far the worst problem that players had, above all others, was that the first hypotheses they formed about the system were not changed in response to the data.  If anything, the players were apt to be the most sure in their beliefs when the hypotheses were completely wrong.  Part of this came from uncertainty and cognitive dissonance.  Uncertainty produced fear and doubt.  Asserting the hypothesis helped quell this fear and doubt.  Over time, the players learned that the more they believed in the hypothesis, the better they felt.  This reduced their ability to develop new hypotheses, as they were already "wedded" to their existing ideas.

The Successes

There were also commonalities amongst the players who did well.  The players who did well were the ones who could tolerate uncertainty.  They defined clear goals and priorities.  They made many small decisions in different areas, and followed up on the expected vs actual results of most, if not all of those decisions.  They kept an eye on the overall processes of the system, and did not succumb to flow experiences.

Adding to this, Dorner's team ran an experiment with two groups of fifteen players each.  One group was drawn from the student population.  The other group was made of senior managers from large industrial and commercial firms.  The managers did significantly better than the students on every possible metric; given several different challenges, they responded appropriately to each one.  Dorner's team was unable to determine if this was innate talent or the benefit of years of experience.

A Decision Making Model

So what is the right thing to do when faced with a complex situation? Dorner presents a possible schema for problem solving, intended more as a helpful aid than as a representation of how people actually solve problems.   

1. Formulation of Goals - deciding what it is that needed fixing, and putting priorities on those goals.
2. Formulation of Models - determining the internal workings of the system.
3. Prediction and Extrapolation - determining the eventual output of the system.
4. Planning of Actions; decision making, and execution of actions - feeding input into the system.
5. Review of effects of actions and revision of strategy - determining the expected model vs the actual model.

Interestingly, this is very close to the Plan/Do/Check/Act cycle proposed by Deming -- it assumes incomplete knowledge of a complex system and tries to improve understanding of the underlying model through repeated iterations of the cycle.

Dorner also notes that being simply told of a decision making strategy did no good at all to players; when given instruction on dealing with complex systems, the players thought that they had been helped and were better able to discuss their failures with better terminology... but their actual performance was the same as the control group.  What really helped players, overall, was repeated exposure to complex systems.  Showing was not enough; they had to experience their own reactions and build up their tolerance to decision making in the face of uncertainty and emotional stress.

Another presentation at Five Minutes of Fame. Christina took video of the talk, so I might be able to put that up as well.

This one took me a while to go through. I picked up a number of books on the subject when Borders shut down, and the talk gave me the impetus to crank through them. The most relevant books were Plan B 4.0, The Ecotechnic Future, and Hot: Living Through The Next Fifty Years. I also recommend The Windup Girl or World Made By Hand for a much better sense of what the future may feel like.

Part of the reason I went through the books was to see how likely the Singularity is. As far as I can make out, it's very dependent on how much free energy is available in fifty years. With a strong global economy, all the parts and dependencies sorted and the sheer power requirements it would take to jam a human-equivalent neural network down through silicon pathways, it's technically possible to have AI. But then you've still got the supply chain management to work through before you can develop more advanced AI, and the overall "cost friction" means that, in practice, AI is only going to be as intelligent as is economically reasonable. Fundamentally, silicon AI is a luxury for the rich. As such, it doesn't show up much in the slides, but I didn't have time to do it justice. (This doesn't go into the biological AI depicted in Starfish, but that's another talk for another time.)

The reaction from Noisebridge wasn't quite what I'd hoped with this: most people found it really depressing. I was a bit surprised, because I had made a concerted effort to scale down some of the more apocalyptic predictions and pointed out that the US makes out way better than most other countries (pretty simple reason: the countries that get hit the worst are the poorest).

Still, it was worth it to do the research, and I got a couple of compliments and a fun conversation afterwards. The future is a large and complicated subject, so I'm going to be going back to the presentation and filling out bits as I find out more.

EDIT: The videos of Hot: The Next 50 Years and A Really Inconvenient Truth are worth watching as well.

[These are the notes for the talk I gave for Five Minutes of Fame at Noisebridge. Slides are down below.]

So this is Five Minutes of Zen. Zen's a topic that everyone knows something about (at least in San Francisco), but it may not always make sense. I'm going to say some things that are broadly true, but isn't the be all and end all -- if you want to really learn about Zen, having a good teacher is still the best way to go about it.

So where does Zen start?

In the present moment.

Zen starts with what we have right now. You're sitting in a chair. You're watching some guy talk. And you have thoughts flowing through your head.

What are those thoughts about? Well, anything really. you could be thinking about what you're going to do next. Or you're thinking about what you were doing earlier. You could be thinking about space aliens or world peace. But everything that we experience, our entire lives as we experience them, are made out of these thoughts as they happen. And one of the great contradictions in Zen is that in order to understand Zen, you may have to do a whole bunch of thinking to understand thought.

Zazen

Zazen is described literally as the practice of Zen. It's meditation, basically. You sit on a cushion and breathe. You are being asked, literally, to pay attention to the present moment, and to be aware of the thoughts inside your head as just that: thoughts. You do this for long enough, and some of the practice of Zazen carries through to the rest of your life. You get into the habit of paying attention to what happens, moment by moment, through your day.

Because everything in Zen starts from the present moment, it has a different idea of reality. Zen doesn't think reality is composed solely of thoughts. But what you look at when you look at the present moment really closely is not objects, but events, all streaming by impossibly quickly.

Form is Emptiness and Emptiness is Form

The Heart Sutra is the most important statements in Zen. It's subtle. On one level, it says "Everything is made out of everything else." On another level, it says "Reality is Perception" -- light and air hits your body and you create an idea of the world from that. On another level, it says "Every object is an event, and every event is an object." You can look at a very fast event like a wave on the beach and see something static, unmoving. Or you can see something like a human life and see a very slow event taking place. Everything is change.

No future, no past

When you sit and look at the present moment, you don't see "future" or "past". You can't get to them from where you are. If you try pointing at the future, you end up describing your idea of what the future will be -- the thoughts and expectations that you have in your head. Same with the past: it's what you remember right now.

No self

When you look inside your head right now, you see thoughts. Some of those thoughts are attached to other thoughts, and get wrapped up in a big ball of thought that we call "self." There are expectations about what "you" will do in the future and how you will act. But that big ball of thought doesn't stay static: your idea of yourself is made up of habits, beliefs, preconceptions and ideas that get swapped out all the time.

Life is Suffering

Suffering is a bad translation of "dukkha", which Wikipedia puts as "the cart with one wheel that is slightly broken, so that the rider is jolted each time the wheel rolls over the broken spot." And the interesting question is -- what makes that uncomfortable? When you slow it down and look at what goes on, it turns out that what really pisses people off is that it just happened, or it's about to happen very soon. Our minds cling to it. What really causes suffering is not so much external events, but the difference between reality as it's happening versus how we'd like it to happen.

You Are The Universe

Zen's very practical. It's not saying a single human body is literally equal to the sum of all matter and energy in the universe. It is saying that the distinction between "this human body" and "the rest of the universe" is an arbitary one. You can't have a human body without the universe supporting it. And interestingly enough, you can't have the universe without that human body.

Because in Zen, what makes a human being isn't just the meat and bone. What makes a human being is also the thoughts, ideas and events happening around him as well: we absorb and give back to our environment. When you are with someone, their idea of you is partly you, and you are partly their idea of you. When you die, parts of the ideas that make up you still float around attached to other people, or in books or Youtube articles. So when it comes to reincarnation -- while it's unlikely that all the ideas in your head will come together at once again, it's not impossible for someone very like you to arise from society. Thoughts are fungible.

Zen's view of individual identity is more than just fluid -- human beings are like rolling waves. We think we exist in isolation, but we mix with our environment all the time. Again, there's no "self", just thoughts. If you want to point to "me" -- don't just point to the fleshy bone bits. Point to your family and friends. Point to your iPhone. Point at Noisebridge. Then point at the sky.

Enlightenment

The usual stance in Zen is that Enlightenment is an event. You realize the present moment, you're free of preconceptions, just for an instant... and then your time is up.

From a neurological perspective, there's no such thing as free will.

I'm not even joking.  The people who work most intimately with the brain -- the neurobiologists, the cognitive scientists -- tend to think "there's not a whole lot of free will out there, and if there is, it's in the least interesting places." [1]  If you read about the workings of the brain, it discusses the frontal lobes, the "executive body" of the brain which makes decisions from the evidence presented to it. [2]  If you read about the process by which brains tend to make decisions, it's a combination of a darwinian "war of ideas" (literally, ideas fight it out in the area for control of the cortex) and pattern recognition -- the brain picks out patterns that it recognizes from a sea of information and will see those patterns until something better makes it pry those patterns loose. [3]

Philosophically speaking, we have free will in that we can choose what we want to do.  The catch here is that we can only choose to do things that we want to do; we choose to do that which is in our nature.  And we don't choose our nature.  We don't decide to be gay or straight.  We don't decide that we're extraverted or introverted, timid or brave, anxious or confident.  We simply are these things.   And while making decisions against our nature may enable us to become more comfortable outside our limits, it's something that we have to push ourselves against, again and again, before we can say our boundaries have shifted.

The fact that drugs can change the way we think is proof that our idea of free will is idealistic.  Forget prozac and ritalin.  We know that you can make your boundaries and the way that you think with enough tweaks to serotonin.  But we alter our mental states daily with alcohol and caffeine.  Our idea of what we choose freely is moderated by "I was drunk" or "I was on my fifth espresso."  If it's possible to be "someone else" so easily and so casually, does it make sense to talk about an ideal, unadulterated self at all?   When you talk about yourself acting of your own free will, you're talking about you with a healthy unperturbed brain.

This also means that "free will" is not a binary switch.  It's more of a continuum.  At one side of the spectrum, you're focused and aware in the moment, choosing your words, recognizing new options, picking things up.  You know that everything you do right now matters.  At the other side of the spectrum, you're running on autopilot.  You may be drunk, tired, caught in the past, or recognizing a false pattern.  You may have your buttons pushed.  But you're predictable.  You don't have any creativity or self awareness.  You can't learn.

So what, in the brain, causes free will?  It's the same brain, right?  What causes the brain to be worn out, or to be fresh?

There's a series of experiments done on monkeys that tells us what makes monkeys and mice able to learn, whether it's finding their way through a maze or finding a raised platform to avoid being dunked in water.  They find that when animals are raised in a stimulating environment, they do much better at solving puzzles.  When they looked carefully at the brains of those animals, the researchers found that their brains show evidence of neurogenesis; new brain cells were found in the hippocampus.  Meanwhile, animals that were not raised in a stimulating environment (or worse, were raised in a stressful one), showed little or no neurogenesis. [4]

If you abstract the research and apply it to humans, it's reasonable (although complete speculation) to argue that human beings will show also evidence of neurogenesis given a stimulating environment.  Certainly it's known that humans raised in a stimulating environment do much better than humans raised in an unstimulating environment, and reducing or eliminating stimulus to humans is considered to be a form of torture.

The interesting thing about tying healthy brain function to neurogenesis is that widely known mental practices suddenly reveal themselves to have a biological basis.  Sleeping eight hours a day.  Regular exercise.  Play.

The function of play in neurogenesis is especially interesting, because play involves the ability to recognize patterns and make quick decisions without consequence to the "real world".  It gives the brain room to "limber up" and build up extra neurons that can be used later.  Those neurons give us the ability to see things in a new way.  It's freeing.  You don't know who or what you'll be at the end of the game, but you know you'll be someone else.  It's an enjoyable sensation and an end in itself.   People who have brains like this have the most free will: they can make decisions taking multiple factors into account with more mental resources available to them.

This also explains the biological aspect to being burnt out -- work too hard, or too long, and you'll use up the store of fresh neurons in your head.  People who work too hard or too long are unable to see new patterns or respond appropriately to unexpected stimuli.  Their free will has been compromised.  They don't just look like being zombies; they are zombies.   

I always thought about free will as important, weighty stuff, worthy of a philosopher.  But it turns out that free will comes into being when we are at play.  It's only after that that we get back to work with our fresh neurons, and by then it's already set in.  

Free will is the ability to be silly.

I gave a presentation at Five Minutes of Fame tonight.  I had to skip over the last few minutes, but all the links and slides are included below.

I'd like to thank everyone for reading the blog post.  It got far more attention than I was anticipating, and while I'm still processing everything, I'm touched by how much people "got" what I was saying and responded positively to it.  There are a number of people out there that have had the same experience (or close to it) that I had, and while it's not exactly fun to admit vulnerability, I'm happy I did it just to hear from you guys.

So, on with the show.  There were a number of responses either in blog comments, or on Twitter (holy god Twitter exploded, thanks @buzz and @tenderlove), or on Hacker News and Reddit.  I thought about responding to comments individually, but it makes more sense to write it up on the blog itself.  In no particular order:

“You weren’t really doing pair programming”

This is an easy statement to make, and a lazy one.  There were a number of engineers there, all of them very competent.  Pivotal Labs had gone through the space on at least two occasions. They had several Pivotal Labs alums on staff, and the bulk of the programming library was Agile and XP books.  They all believed this was pair programming.  If that wasn’t pair programming, I don’t know what is.

I anticipate the response to be "well, it wasn't done the way they say to do it in the books."  Yes.  That's because nothing is done the way it's done in the books.  Ever.

EDIT: And as far as I can tell, they were working directly from the book in that "all significant development is done in pairs" and "If you prefer to work alone, that's fine.  You just can't work with us."  People who are saying pairing is a part time or optional activity -- are they also doing it wrong, or just tailoring it to their team?  Different practices are agreed on for different teams, and this was the practice they chose.

“You walked into a situation where you didn’t know any of the tools.  What were you thinking?”

I was thinking that pair programming would be an excellent way to be mentored on the appropriate use of those tools.  I was upfront about my lack of experience with those tools and front end programming in general, and was looking forward to learning about it.  Bear in mind I wasn’t going in cold: I know Ruby very well and knew at least the principles of jQuery, while knowing enough CSS and Javascript to get by.  But you’re correct: it was a large part of the problem initially.  

Something I touched on briefly in “No high notes”: the best way for me to get the feel of a framework is to sit down and work with it.  I’ll read through the manual, build a couple of example applications and push the limits of what it can do.  I’ll read through the source code, sign up to the mailing lists, find a couple of embarrassing bugs, and I’ll know the shape of it in my head.  And that knowledge is solid: I can tell you some fine details of a framework even if I haven’t touched it in five years.

I couldn’t do that when pair programming.  Lectures and presentations tell me nothing about what the first hand experience is: it’s like thinking you know how to ride a bicycle because you saw it done on Youtube.  Time after time, my partner would happily dash out a complex jQuery statement pulling out an element by its div and class attributes, and I’d then struggle to do something similar while I was still figuring out the syntax.  Every day, I’d get more and more of an overall larger picture of the system, but it was through the corner of my eye as we were whizzing past to our destination.

“You should have spoken up more.”

That's a fair question.  Why didn’t I speak up more?  I asked co-workers about my general experience and raised some concerns, but I didn’t push nearly as hard as I could have.

The best answer I could give is that I expected it to be hard at first, and then to get progressively easier.  At the point where it wasn’t getting easier I was concerned, but I wasn’t going to tell them they were Doing It Wrong.  Remember that I was coming into pair programming with a very different background, and was there to learn.  I did speak up when it came to the design and refactoring issues I saw, but when it came to XP I had no basis for comparison.

I did tell some of my partners to please stop grabbing the keyboard and suggested that I could tackle problems that didn’t require a pair, but XP is fairly clear that pair programming is the rule, not the exception.  XP is also clear that this way of working makes programmers happier and more productive.  Ultimately, I wanted to believe it.  I told myself my own experience was wrong and that the hundreds of books, blogs and comments were right.  I kept quiet.  

“You knew pair programming was what they did and you had an interview where you pair programmed: how did you not expect this?”

Also a fair question.  The truth is that I didn’t know what to expect, and nothing that I read in the literature lead me to expect that I would dislike it as much as I did.  The initial interview was, by necessity, done in a couple of hours with a single person in another room.  The following day of pair programming had to be cut short due to a misunderstanding with the recruiter: we’d only scheduled an hour.  The problem that was supposed to have taken a day was mostly fixed inside of that hour, and we all enjoyed the experience.

“Have you considered autism / you must suffer from social anxiety.”

No, I suffer from introversion. (Although, some might say I suffer from extroverts.)  Put simply, interaction with people tires me and drains me of energy.  Being in a quiet room with one person is fine.  Keeping an active conversation with background noise or a television is a noticeable drain.  Keeping a conversation up with several people at a party where loud background conversations are happening is a larger drain.  And about thirty minutes in IKEA is enough to turn me into a zombie.

I can certainly behave in autistic fashion when I’m low energy, but it’s not my default state.  And social anxiety is something that everyone is prone to, especially when things don’t seem to “work” but it’s unclear exactly what to fix.

“Pride in ownership is a bad thing”

It wasn’t the best phrase I could have used.  Pride in good craftsmanship would have been better.  I have absolutely no problem with people refactoring code -- just because I wrote something and it worked for a particular problem at a particular place and time doesn’t mean it can’t be made better.  Seeing how people looked at my code and did one better is actually one of the perks of the job -- the day I stop learning is the day I hang my hat up as a programmer.

“It was a dysfunctional / deathmarch environment.”

Actually, no.  While it was a dysfunctional environment for me personally, most of the people there certainly seemed to be very happy.  It’s worth reading my original thoughts for when I first joined: it was about as far away from a death march as you can imagine.   Bear in mind that there were 10 programmers on the team, and we switched daily.  It’s not like I had one bad partner: it didn’t gel with any of them. 

They were doing a UI redesign of the entire front end, and had a hard deadline based on the engineers they had at the time.  They had evidently spent much time determining the tools and wireframes involved in the redesign.  For many of the engineers there, this was a long overdue opportunity to sit down and really crank.  Some of them were amazingly intelligent and ambitious people, as good as any I’ve worked with.  Pairing with me must have been a frustrating experience, especially for the younger developers who were not used to being mentors.  

For my part, I was frustrated by my own lack of skill working on the front end, and was keenly aware that I was slowing people down every time I asked a question.  Rightly or wrongly, we had a job to do.  No matter how patient and giving they were, ultimately we were being tracked on our points delivered.

And furthermore, I think this is a lazy response.  It’s the moral equivalent of having a user file a bug, assuming user error, and then closing the bug as WORKSFORME without actually looking at the code.   

Bear in mind I am not saying that their approach to development was flawed in principle or even in practice -- I was but one engineer out of many and they had to consider the entire team.  Bear in mind that this company followed XP down to a very fine detail and prided itself on its software process.  Bear in mind I joined this company specifically because I wanted to work in a company that did Agile.  I would have left immediately if it had been obviously dysfunctional or a deathmarch.  

“Why were they pairing if they had a fixed deadline and needed to crank out work?”

Good question.  Given the context and the relatively straightforward work that they were doing, I believe that the project would have been completed faster if they had not been pairing for the duration.  It may have been slightly lower quality, but it could have worked.  It may have been that they judged the benefit of always pairing to be more valuable that any short term gain.

But honestly, my guess is that the question never occurred to them.  Pair programming is a core practice of XP.  If you’re an XP shop, that’s what you do.

“Pair programming can produce amazing work”

This is something I don’t dispute.  Almost everyone I know, myself included, has done pair programming at some point and had positive results.  Even Dijkstra pair programmed and found some use in it.   

However, that’s very different from saying that long term pair programming is flat out better, which is what XP says.  Most studies find little appreciable difference between the work that a pair does, and the same work done individually.  Anecdotal evidence is not reliable in this field because humans are subject to all manner of cognitive bias; Hacknot (an under-appreciated website that is now an ebook) does a wonderful dissection in Anecdotal Evidence And Fairy Taies.  I’m not saying it’s a worse practice in general.  But a meta analysis shows that there is no statistical evidence that this is a better practice overall.

Leaving that aside, there’s the small problem that programmers are not fungible.  The best programmers are up to 28 times better than the worst programmers.   Even if pair programming were proven to help the average programmer, it won't work for some edge cases.

Tying a good programmer to Guy Steele or Richard Stallman will make the good programmer slightly better, but will drastically impede Steele or Stallman.  Even tying Steele and Stallman together (or even better, Stallman and Gosling -- they're both Emacs fans, right?) wouldn't produce the output of a Voltron-like superprogrammer (EDIT: I am wrong.  They did, and it did.).  And while I know some people who would love to be paired with Zed Shaw, Jamie Zawinski or Hani Suleiman, I’m fairly certain there are others who wouldn’t survive the experience.

The Point

This is my point: One size does not fit all.  Individuals and Interactions over processes and tools.  And if you’re considering implementing a process, think about your team and consider what works for them.  As Elisabeth says, introverts need attention too.

The last time I wrote about Agile, I was writing about Grockit, and their daily process.  It’s been almost a year, but I think I have enough perspective now to write about my experience pair programming.  

EDIT: I've read through the responses and replied here: http://tersesystems.com/2010/12/31/responses-to-where-pair-programming-failed-for-me

The short story is that pair programming doesn’t work for me as the main way of developing software.  I can pair program for a day, or maybe a week, especially if we’re focused on a particular problem.  But after that?  I’m done.  Toast.  I don’t want to see anyone, talk to anyone, and I need at least a couple of days in a cave until I’m fit for human company again. 

It’s a sad story, but the funny thing is that I’m so much happier now with how it ended.  I’m happily employed on a contract where I work from home or from a coffee shop, and I’ve made new friends and explored more of San Francisco than I ever thought possible. I have a bicycle and a laptop, and as long as I meet my deadlines and check in code regularly, my time is my own.

I’ll list the big problems I have with pair programming up front and give you the detail and anecdotes later. 

1. Split focus.
2. No experimentation.
3. No high notes.
4. No pride in ownership.  
5. No escape.

Background

My background is as a backend programmer.  I’ve done inventory systems, credit card processing and fulfillment, transcoding and digital asset management.  In non-agile teams, I was used to being handed the largest, hairiest problem in a project, pulling out the domain and putting something solid together inside the deadline.

I came into Grockit at the tail end of a long period of discussion where they had decided to rewrite the entire front end UI.  They were making heavy use of jQuery (a Javascript AJAX library), SASS (a CSS variant) and a library called Erector, which produced HTML markup from Ruby classes.  I knew none of these things; I knew some Javascript and CSS only from implementing them on my blog.  Because they were focusing on the UI and they were on a tight deadline, they were going to do an eight week sprint, and rewrite as much of the UI as they possibly code in that timeframe.

So, my first day.  

I was watching a man looking at a page -- what the finished product should look like.  He was telling me they’d discussed what the features should be already, and we just had to bang this out inside a day.  

He had opened up a text editor, and we were looking at Ruby code.  The ruby class was actually a page composed of Erector methods -- ruby code that evaluated to HTML -- but it had been refactored into extremely small methods which called each other, so there was no way to ‘see’ the HTML and how that corresponded to the class -- you simply had to know it.  In addition, they had tests attached to each section of UI code.  If you refactored the UI, you had to rewrite all the tests.

Sitting with another man, trying to work together to produce working code.  I was shocked at how unnatural and unintuitive it felt.  Far from being an easy practice, it felt both clumsy and shockingly intimate; both too personal and impersonal.  

If I asked a question, then work stopped.  

If I asked about the background of the code or the design, then work stopped.  

If I tried to follow along, then I found that I was trying to guess at what various methods and frameworks did.  I was being bounced between the domain classes, the Erector classes, the Javascript and the various tests, and I didn’t have a grasp on any of them.  

I understood the HTML code, but even there, I didn’t even know what to say -- did I point out the typos?  The two methods that were almost exactly the same except for the innermost loop?  Was I nitpicking by pointing these things out, or signalling my involvement?

And then, he was done.  It was time for me to write some code.  I opened up some files, looked around at the methods.  Tried to figure out what was going on, and write some exploratory code to see what was happening...  And then my pair started typing the method I was trying to write, handing it back for me to test.  I found that this would happen if I paused too long, or seemed to be typing the wrong thing -- my pair had another keyboard, and he would type over me to try to “finish the sentence” rather than talk to me. 

At the end of my first day, I had a massive headache from trying to absorb everything.  I was told I’d done very well, and that it was a lot to take in on the first day.  I didn’t feel like I’d been very productive, but I hoped that we could settle down into some kind of flow.

Split Focus

Split focus happens when I try to do two things at once.  Human beings are not good at multitasking in general, and I’m worse than most.  Pair programming is a balancing act between managing code and managing the pair that you’re working with.  Go too far in one direction or another, and you both fall off and lose effectiveness until you restore balance again.

I found that in order to pair, I had to act as if I was in a continuous meeting.  I had to not just listen to my pair, but appear to be listening; I had to nod in the right places, repeat back what my pair said in active listening fashion.  I had to pick the right moment to interject.   I tried to model my partner’s mental state in my head so I could see his viewpoint better. 

While I was doing this, I was trying to see the code that he was writing, and the design that he was trying to make the code fit.  If there was a failing test, I was trying to figure out the test and the test framework at the same time.

And if I was writing code, I found something very interesting: I don’t think in English when I write code.  I think in code first, and I can translate written code to English.  When I was trying to talk to people when I was writing code, I found that I’d have to write pseudocode and then talk to them about it -- and when my pair wanted me to talk about code, he wanted me to stop typing.  But if I stopped typing, I couldn’t describe what I was doing.  Every time I tried to write code and talk to my partner at the same time, I could feel the lurch between what I could feel -- the complex shape of it in my head -- and what I was having to say.

So pair programming split my focus not just in one way.  It split it in three or four different ways, and kept it split.  No wonder I had a headache.

No experimentation

The large part of pair programming was doing things in “The Simplest Possible Way That Could Work.”  In practice, this meant doing it the way that it had been done previously, with the least possible amount of change.

The way that the Erector toolkit was used, combined with pair programming and test driven development, ensured that two Ruby programmers were required to change any and all elements of the UI.  Tests were written to check for every single element in the UI, and with good reason; because elements of the UI had been abstracted out into subclasses and views, there was no clear way to see what code would do until it had been fully executed.  Sometimes the simplest possible thing to do had been to hack one of the classes for a special-case logic.  Then more code had been put on top, until a refactoring was just too expensive.  As long as all the tests were passing, the code was in a known, "good" state -- and we had to keep it that way.

The simplest thing that could possibly work as far as I was concerned was to put together an ERB page.  A straight HTML page with some embedded Ruby was a known, easy solution that would have enabled straight web designers to work on the UI without involving the full programming team, or at least have allowed us to quickly compare the mockup with a page.  But that ran into another problem; there is no facility in pair programming to change tracks, or try anything different.  Spikes only happen at the beginning of an iteration, and this iteration was budgeted at 8 weeks -- we were locked into our current way of doing things.

Not that it mattered; there was simply no leverage to explore the framework or try different approaches.  Any code changes that weren’t immediately applicable to changing the current page were “going off track”.  I couldn’t see the parameters of the current system, or feel for the fragile points if it didn’t match up with what my pair thought was important.  Every single moment I was typing, I was being watched.

No high notes

When I got to the point where I understood some of the framework, I saw that there were some bugs inherent in the existing design which would make the system act unpredictably in production.  In other situations, the design could improved by eliminating some classes and moving some other classes to have a single responsibility.

To me, this was “refactor mercilessly” and “once and only once.”  But to my partner, this was a violation of “do the simplest thing that could possibly work.”  It also went against the daily switching of partners.  Whatever you start when you pair has to be finished, and checked in, by the end of the day.  If you’re only looking at getting the story done by the end of the day, you’re not interested in hypothetical bugs or refactoring -- you don’t have time for them and they’re not budgeted for.  

The only way I could move discussion along was if I could write a test for it.  If I couldn’t write an effective test, or I described it in unfamiliar terms (usually from books or blogs), he wouldn’t hear “solves a problem that is difficult to test but is real.”  He would hear “introduces needless complexity and complicates design” and point to the simplest solution that makes sense to him.  If I tried writing a test and it took too long, I lost there as well. 

Eventually, I realized that many of my partners had never worked outside a test driven development process: they didn’t have the same idea of design or architecture as patterns distinct from code.  Trying to argue for encapsulation or adherence to SOLID principles is pointless if your partner has no background, let alone hands on experience, with what you’re saying.  This goes double when you’re talking about hard-won domain expertise: the more I knew about a subject, the less I could say.

I started picking my battles.  

How you pick your battles is, you plan to lose in the best way possible.  You try to leave an opening so that the bug that you know, from past experience, is going to hit, is not going to result in any lost data and can be easily fixed later.  It’s not great, but at least you can recover from it.

There are no high notes, no thought out design.  There’s only the design that could have tests written for it, in the time you had to write the tests.  The classes look reasonable at first glance, and the methods are seemingly bug free.  But the cracks are there, if you know where to look.

No pride in ownership

There is no ownership in pair programming.  You’re working on a different piece every day, with a different partner.  Everyone owns all the code equally.  No-one really has responsibility for a single piece of it.  Pair programming views this as a strength.

I had no pride in ownership.  The code that ended up on the ground every day wasn’t something I felt I had input into.  It didn’t feel right to me.  Doing a good job matters to me.  My idea of a good job: years after I’d moved on from one project, someone I’d never met wrote me a thank you note for taking the time to make my code clear and easy to work with.

The part of my brain that works though code will chew through things.  Impede the flow, and that part of my brain starts to itch.   Block it, and it starts pounding, chewing away at anything that comes close.  Without pride in ownership, there was no meaning to any of the work that I was doing.  I couldn’t even identify it as mine.

It was about this time that my physical health started to pack up.  Winter set in, and it started raining constantly.  I got a cold that got into my lungs.  I couldn’t climb the stairs without resting.  I couldn’t write code.  I couldn’t think.  The rain beat against my window.  I couldn’t sleep.

No escape

The worst part of pair programming was getting up in the morning and realizing I had to do it all over again.   No matter how drained I was, no matter how much I wanted to get away from people: I couldn’t.  As an introvert, I yearned to be left alone with a large problem and the freedom to slice it into pieces.  But there was no escape. 

Even at the best of times, there were people.  First, a fifteen minute standup meeting.  Then sitting down at a work station with engineers on either side of me, two to a table.  Talking.  An engineer sitting next to me in my personal space.  Every moment I was sitting there watching him, I was trying to think of something to say.  Whenever I was typing I was worried that I was too slow, or that I was taking too much time to think, wasn’t saying enough, wasn’t being clear.  Was I being anti-social?  Talking too much?  Too much smalltalk?  There was no way to know.

Even when pairing worked, it was the slow, clumsy communication of two people trying to move a large sofa down a staircase.  There was no flow.  There were no spots where there was no conscious movement between thinking something and having it appear on the screen.  I missed it horribly.  Even the small discussions: was having “as_currency” method on an ActiveRecord Model the correct thing to do?  Or was it something that should belong to the UI, as a helper method?  Just how much did I really care?

And then a group lunch, where there was food, small talk and team bonding experiences.  I started stealing moments of solitude whenever I could.  

I asked my co-workers if they saw what I saw, if I was missing something, anything -- I didn’t see how this could work, how people could keep doing this.  They said I was doing fine, that it just took time to settle in and adjust.  That it was hard for everyone at first.

Eventually, I retreated into myself.  Between the blinding headaches, the insomnia, and the pounding, unmet need to write code, I stopped responding to input.  I could stare at a screen and not see anything.  Someone could talk to me unexpectedly and I wouldn’t hear them.  I was fulfilling the rote requirements of my job, but I wasn’t there.  I’d used up everything I had just showing up for the day.  I started checking my iPhone when my other partner was typing.

Finally -- just shy of three months later, and for the first time ever -- I was fired for not being a team fit when pair programming.

Not Alone

I wrote this not just to understand it, but also to be able to talk about it.  There’s been a presumption that pair programming works for most people and is much easier and faster than programming solo would be.  This may or may not be the case, but as a long term practice, pair programming doesn’t work for me.  There are many other people that pair programming doesn’t work for either.  We matter too.

Mark Wilden had a similar experience at Pivotal Labs. His discussion of design rings very true.
Nick Carroll says that pair programming is not for him. He thinks peer review is more useful.
William Pietri has a list of anti-social practices, many of which I saw and/or practiced.
Software Reality has more to say about the XP approach to pair programming and design.  He also has a seperate post addressing pair programming specifically.
A CodeRanch discussion also discusses issues with pair programming, including Damon Black’s experience.
More generally, Daniel Markham talks about some gaps in agile as it is practiced and has made an interesting list of the responses.

EDIT: Also, Kathy Sierra writes about her experiences with pair programming in Pair Programming is NOT a choice, complete with pre-filled in responses.

EDIT: Another programmer describes his experience and gets a follow up response here.

EDIT: Another article: Nobody Expects The Agile Imposition.

EDIT: Alex Ruiz discusses his mixed feelings about pair programming.

EDIT: A story from Xebia about Chris, the tester.

EDIT: Jay Fields talks about why he doesn't pair program any more.

EDIT: Adam Logic discusses his dislike of pair programming and compares it to Design By Committee.

Also, it’s important to note that I still believe that Agile development can lead to better business solutions with less wasted effort.  Pair programming is part of a single agile process called XP, and there are many other Agile processes such as Crystal Clear which do not mandate pair programming as a daily practice.

EDIT: I've read through the responses and replied here: http://tersesystems.com/2010/12/31/responses-to-where-pair-programming-failed-for-me