Pair Programming or Bare(ly) Programming

image

“Sorry we don’t have enough resources, we only have four pairs” – As an engineering leader no other statement has made me cringe more.  After all four pairs is a healthy sized team of eight developers. 

Throughout my career I have run across CTOs, VPs, directors, development managers, teams, and individual developers who swear by pair programming with near religious devotion.   Personally I’ve maintained a healthy dose of skepticism when it comes to pairing as an overarching development philosophy.  

As an engineering leader my job is to build products that delight customers in the most efficient way possible.   Anecdotally, pairing consistently costs more and hence seems irresponsible to use exclusively as a development technique.    But admittedly anecdotal evidence is insufficient so I decided to dig through the research and see if I could find more empirical evidence to support my claim.

Background

Pair programming is an agile software development methodology where two programmers work on the same task using one computer and keyboard.   One programmer is called the driver and operates the keyboard and does the primary coding work.   The other developer, often called the navigator, is responsible for observing the driver and providing guidance in order to speed up problem solving, improve design, and minimize defects.

The potential negative impact of pair programming is immediately clear to most people.   By applying two resources to a task you are effectively doubling the cost.  So unless there’s an equal or greater improvement in other project variables, pair programming would be nearly impossible to justify. Exploring the problem through a project management lens, where we have three variables, cost (including resources), time, and quality/scope, If we double our cost we’d expect to see an equivalent decrease in time to deliver or increase in quality or scope (or some factor of each).

image

In mathematical terms let’s assume the value of any given project X is equal to a weighted linear combination of cost, time and quality/scope. 

image

When pairing our cost is automatically going to double since we’ve applied two resources for a task that in theory can be completed by one.

image

In order for our project value to remain equal or be better we need our other variables to proportionally change in the right direction.   For example if our project now takes 50% less time we could argue we net out even.  Or if our scope or quality double, we would similarly be in a good position.

image

However, In my experience I’ve not seen pair programming live up to these expectations.  Instead I’ve seen tasks or user stories take the same amount of time and produce similar results at nearly double the cost.  But you shouldn’t take my word for it.  Let’s review the literature and see what the experts have to say.

Research

There are actually a fair number of research papers that attempt to prove or disprove the efficacy of pair programming.  That said, in my survey of the literature I found most of the research to be ill designed for comparison to real world corporate product development organizations.  Specific issues include:

  • Developer Skills:  Most of the studies rely on university students that shouldn’t be compared to seasoned professional developers.
  • Non Production Environments:  The majority of the software used for evaluation is very far removed from real product development environments.
  • Organization Realities: Finally there is little or no accounting for organizational churn that happens in a real for-profit company
  • In spite of these issues it’s worth exploring these various research studies and the insights they provide on the impacts of pair programming.

    Many of the research papers evaluate the impact of pair programming on effort, which in at least one paper is defined as two times the duration or time required to complete a given task [1].  Specifically, effort increases ranging from 15% all the way to 100% have been observed [2].  In one of the more well conducted studies an effort increase of 84% was seen [1].   Since we know effort is just twice the duration of a single developer we can actually do some math to figure out how much faster pairs complete a task versus a single developer.

    image

    Or by using our earlier project management equation, with a little rounding we can assume our pairing time weight would be roughly 9/10 the weight required for a single developer.

    image

    This is nowhere near the factor of 1/2 or less we said we needed to make pair programming cost efficient.  Well if the research doesn’t support a sufficient decrease in time to completion perhaps there’s research indicating that a given project’s scope or quality will increase enough to offset the difference.  

    Unfortunately, once again the results are at best inconclusive, but in many cases support an actual decrease in scope and minimal or near zero increase in quality.  For example in [2] a reported 29% decrease in productivity was measured for pair programming team when measured as a function of completed use cases.

    Regarding quality, even in one of the more optimistic papers we only saw a 10% – 20% increase in quality (measured as test cases passed) [3].   According to [2], we only saw an 8% improvement in quality when measuring actual defects.   While these improvements are non trivial, when combined with the time and scope metrics it remains insufficient to offset the associated costs.

    Cherry Picking

    “But aren’t you just cherry picking the worst examples to justify your case” you might ask? Not really because even in the most optimistic research studies initial results were usually much worse and only improved over time.  For example in [3] initial increases in effort dropped from 60% to15% over time. Most of the research attributes these gains in effort to “pair jelling”.  In other words, as the pairs get to known each other they become more efficient.

    The problem with these studies is that they assume that once a pair jells the gain will hold.  However in any real for-profit organization there is potential for high variability in projects and staff which means pair jelling is unlikely to be a one off cost.  It is more likely a continuing cost to the business over time.

    Several studies also point out that the value of pair programming decreases with simpler tasks [4].   Therefor one must consider the ratio of simple to complex tasks in any given development cycle in order to understand the long term impacts of pair programming.  When I evaluated my own teams, I found multiple iterations where 75% of work items where smaller changes that could easily be tackled by a single developer in the same timeframe.  

    Finally, one paper [5] attempted to justify pair programming by evaluating Net Present Value (NPV).   In this paper an argument is made that even if it costs more to pair program, faster time to market warrants the cost.  I take issue with this calculation since it does not factor in the opportunity cost of having those extra resources not work on a different higher priority project.  

    For example if we take the reported 84% increase in effort and assume we finish our project in 9/10 the time of a single developer, we must ask ourselves what happens when a key customer asks for a critical bug fix?   I can tell that customer to wait until I finish my current project or I can split my pair and work on both at the same time at the small cost of a 1/10 increase in duration.  By splitting my pair I’ve delighted my key customer as quickly as possible at a trivial cost. Clearly you need to factor in the opportunity cost of not delighting that customer when evaluating the value of pair programming.

    To Pair or Not to Pair

    So should you pair or not pair?  There are a lot of reasons a team might use pair programming.  In some cases the cost / benefit tradeoff may be worthwhile.  Pairing can be very effective at educating new team members, improving the skills of junior team members, cross training, and reducing the cost of complex tasks.  If you take anything away from this post let it be:

  • Challenge the Efficacy of Pair Programming: If your team or engineering manager wants to exclusively use pair programming, don’t blindly accept it.  Collect the data to validate if it is really cost effective
  • Pair when it makes Sense:  Use pairing selectively when it makes sense including educating new team members, improving the skills of junior team members, cross training, and reducing the cost of complex tasks.
  • Factor in Opportunity Costs: Make sure you consider the opportunity costs of projects not being worked on when pairing.
  • In short don’t allow yourself to be swayed by a dogmatic insistence that pair programming is better.  As a leader your job is to challenge your team to delight customers in the most cost effective way possible.   Pairing should only be used if it definitively contributes to that cause.

    References

    [1] Arisholm, Erik, et al. “Evaluating pair programming with respect to system complexity and programmer expertise.” IEEE Transactions on Software Engineering 33.2 (2007). – Summary available at https://pdfs.semanticscholar.org/9787/c9663cad3a1c21550f2e5e365e70fd01d3aa.pdf

    [2] Vanhanen, Jari, and Casper Lassenius. “Effects of pair programming at the development team level: an experiment.” Empirical Software Engineering, 2005. 2005 International Symposium on. IEEE, 2005. https://pdfs.semanticscholar.org/40dd/fa666bf367cfffaae421dbd3c6170a3e3dc3.pdf

    [3] Cockburn, Alistair, and Laurie Williams. “The costs and benefits of pair programming.” Extreme programming examined (2000): 223-247. http://www.cs.pomona.edu/~markk/cs121.f07/supp/williams_prpgm.pdf

    [4] Lui, Kim, and Keith Chan. “When does a pair outperform two individuals?.” Extreme programming and agile processes in software engineering (2003): 1011-1011. ftp://nozdr.ru/biblio/kolxo3/Cs/CsLn/E/Extreme%20Programming%20and%20Agile%20Processes%20in%20Software%20Engineering,%204%20conf.,%20XP%202003(LNCS2675,%20Springer,%202003)(ISBN%203540402152)(479s)_CsLn_.pdf#page=240

    [5] Padberg, Frank, and Matthias M. Muller. “Analyzing the cost and benefit of pair programming.” Software Metrics Symposium, 2003. Proceedings. Ninth International. IEEE, 2003. http://wwwipd.ira.uka.de/Tichy/uploads/publikationen/32/metrics03.pdf

    Creating a Management Philosophy

    Just In case you couldn’t infer this from my previous posts some folks consider me opinionated and occasionally dogmatic.  What else would you expect from a born and raised New Yorker, who grew up in a household where arguing your point was considered a cultural birthright!

    Unfortunately, while having strong opinions and ideas can be a positive, I’ve found throughout my career that those ideas don’t always resonate with coworkers.  Even when those ideas are sound.  When I started to lead and manage larger teams this increasingly became a handicap and I soon realized I needed a better way to get my thoughts across.

    Enter Pete Carroll.   That’s right, the same Pete Carrol who led the Seattle Seahawks to two super bowls and USC to two college national championships.  Not too long ago a good friend and neighbor happened to tell me about a management training class he attended based on material from Pete Carroll and his “Win Forever” philosophy.  It sounded very compelling and I immediately purchased Pete’s book based on the same concepts.

    It was immediately clear from reading this book that Pete Carroll had faced similar challenges earlier in his career.

    “But while I had a sense inside me of what we needed, I hadn’t articulated it very well.  I didn’t have the details worked out in my own mind so that I could lay them out clearly and convincingly to anybody else”

    In short, this book preaches a simple strategy for dealing with an inability to convey your ideas, which is to write them down, iterate on them, and formulate them into a single cohesive vision.   By doing so you change the conversation from “hey, here is my opinion” to “hey, I have a strategy for winning and here it is”.   Or in Pete’s words;

    “by December I finally had a clear, organized template of my core values , my philosophy, and – most importantly – my overarching vision for what I wanted to stand for as a person, a coach, and a competitor”

    Armed with a singular vision and philosophy you have a solid foundation to convey your thoughts.  And suddenly you have transformed disparate ideas into a recipe for success.  The implications of documenting your philosophy are huge and by doing so you will:

    • Set clear expectations for your employees
    • Set expectations for executives, higher level managers and peers for how you operate and how it will benefit them
    • Have a recipe for success that you can continually improve and iterate on

    With this information in mind I decided to take years of ideas I had accumulated and started to jot them down.  I refined them and wove them into overarching vision.  And when I thought about what I was ultimately trying to achieve it became clear that I was always trying to deliver truly innovative software to as many people as possible.   And so my Innovate for the Masses™  philosophy was born.   I present it to you below unedited.  It is a continual work in progress but something that has served me well so far.


    Philosophy: Innovate for the Masses™

    Innovate for the Masses™
      • Create unique and defensible value: Products should deliver something truly special that cannot be found in other solutions and simultaneously provide a defensible moat.
      • Build best in class solutions: Products should be fully functional and should not cut corners. We should do everything required to delight customers, nothing more and nothing less.
      • Support all customers: Products should be accessible by all existing and future customers. One off solutions are never okay.
      • Enterprise class reliability and scalability: Product should be robust with 4 9’s reliability and the ability to scale to all customer demands.
    How we do it
    • Ruthless prioritization: We question the necessity and value of every feature or piece of code. We only work on things that deliver essential value to the customer.
    • Avoid premature optimization: we only build exactly what the customer needs. No more and no less.
    • Read between the lines: We listen to our customers but don’t just cater to their demands. We find the commonality amongst all our customer’s requirements and build a truly unique and defensible product that surprises, delights, and addresses their needs.
    • Communicate like crazy: We are one team with one vision and one goal. Everybody must constantly be talking to innovate and build cohesive products
    • Right spot right time: We believe every team member plays an important role in the team’s success whether that is in a leading role or a supporting role.
    • Work harder than anyone else: We will win by out working all of our competitors.
    • Don’t chase the competition: We don’t chase every move our competitors make. We pay attention but follow our vision and goals and methodically work towards delivering on them without being distracted.
    Expectations for our people
    • Be insanely passionate: Our employees exude passion. We are a passion first organization.
    • Get a lot done / execute like crazy: Our employees are insanely productive. They get more done than anyone else.
    • Care: Our employees give a shit. They care about the product, team, company, and customer like something they hold dear.
    • Have a sense of humor: Our employees laugh. At themselves and each other. We believe you should leave work every day having smiled so much it hurts.
    • Don’t whine or complain: Our employees don’t whine or complain they express their opinions and try to instigate change in the direction they want to see. If a decision doesn’t go their way they disagree and commit.
    • Don’t play politics: Our employees don’t play politics. They lay it all out on the table and do their job to the best of their abilities … that is what they get rewarded for.
    • Dare to disagree: Our employees disagree loudly and proudly. Good disagreement is central to progress. Different opinions are valued and we seek out constructive conflict

    Having a vision and philosophy is not all rainbows and unicorns.   Creating a philosophy and broadcasting it to your coworkers is the equivalent of driving a giant metal stake deep into the ground.   You may find throughout the course of your career that sometimes people don’t agree with your strategy and when they don’t you only have three options, change your strategy, change their minds, or move on.  Or again to quote Pete:

    “Coach Seifert was specifically adamant that I not change who I was or my mentality. He said clearly “Pete, you’ve got to do it the way you know how.” After my experience in New York, I wondered if I shouldn’t try to be more political, but the advice I got from the two mentors was uncompromising – and some of the best I ever received.”

     

    In closing if you are anything like me or Pete Carroll I strongly encourage you to write down your great ideas and formulate them into a cohesive philosophy.   It will be will worth your while.

    **For reference Pete Carroll explicitly calls out John Wooden for influencing his strategies and techniques.  I highly encourage people to also read John Wooden’s book “Wooden on Leadership”

    Managing Research Projects in an Agile Development Environment

    Anyone who has worked in an agile organization has found that certain projects don’t quiet fit the agile mold.   Nowhere is this more apparent than with research oriented projects.   After all if there is complete uncertainty in the scope and outcome of a project, as would be the case in a research project, how do you create user stories and estimate story points?   And if you can’t create stories and estimate the associated costs how can you hold your team accountable, communicate status to the rest of the organization, and make cost / benefit tradeoffs?  Simple!  You can’t.

    I’ve personally dealt with this issue after hiring several researchers to work on an agile software product team.  Initially, I struggled to interleave our research projects with our other production work so I started looking for a solution.  The answer to my problem came after reviewing the agile literature and the scientific method and concluding that research projects really just represent an extreme of what the agile process is ultimately trying to solve. Below I will walk you through how I arrived at this solution and details on how you can apply similar tactics in your own research organization.

    AGILE PROCESS

    Early in my career at Microsoft someone handed me a copy of Steven McConnell’s book Code Complete.

    At the time my greatest take away from that book was the concept of the “Cone of Uncertainty”.   The “Cone of Uncertainty” states that the uncertainty of a given project decreases as time progresses and more details are flushed out.

    image

    Historically the “Cone of Uncertainty” was dealt with by creating detailed upfront plans and using waterfall project management approaches.  The trouble with those methodologies is that they’re extremely resistant to scope change.   Largely because scope change reintroduces uncertainty.

    The agile manifesto attempts to eliminate the “cone of uncertainty” problem by following the principle of “Responding to change over following a plan”.   Most agile methodologies use some form of iterative development to reduce uncertainty, with the idea being that if you’re working on smaller well defined chunks of a larger project uncertainty is removed and the project can slowly adapt to changing requirements.  Mike Cohn wrote in an article titled “The Certainty of Uncertainty”.

    “The best way to deal with uncertainty is to iterate. To reduce uncertainty about what the product should be, work in short iterations and show (or, ideally give) working software to users every few weeks. Uncertainty about how to develop the product is similarly reduced by iterating. For example, missing tasks can be added to plans, inadequate designs can be corrected sooner rather than later, bad estimates can be amended, and so on.”

    If I take the above information together I can conclude two things.  First, the agile method attempts to reduce or eliminate uncertainty by making every project a function of smaller work items iterated over time.  Or framed in mathematical notation:

    clip_image002[8]

    Where: T = Max Iterations, M = Backlog, N = User Stories belong to M

    Secondly, if a research project is really just a project with maximum uncertainty then the same framework should apply.   Only there would be an unbounded number of work items over an unbounded amount of time.   Or framed in mathematical notation:

    clip_image002[14]

    According to this logic a research project should actually work within an agile framework.   We just need to figure out how to construct M (i.e. backlog) and how to bound M and T (i.e. number of iterations).

    SCIENTIFIC METHOD

    So what are reasonable user stories for a research project and why are they potentially infinite?  It occurred to me that research in general follows the scientific method and that the scientific method may be a good framework for story generation.

    image

    In essence the scientific method can be boiled down to three phases: a research phase, an iterative hypothesis testing phase, and a communicate or productize phase.  The unbounded component of research is that many hypotheses end in failure leading to another hypothesis that must be tested and this can potentially go on ad nauseam.  This provided me a compelling framework for how to break research into user stories.

    image

    The first story in any research project correlates to the first phase in the scientific method.  This story should be a time bounded spike that frames the initial question, covers any background research, and has an acceptance criteria of generating the required stories for the next phase of the project, hypothesis testing.

    The next set of stories are all part of the hypothesis testing phase.  These stories include any development work required to test the hypothesis, any data collection required, running the tests, and analyzing the results.   If the hypothesis proves false the team should circle back to the background research phase and continue on with the process.

    The final phase in this framework is only relevant when a hypothesis is proven to be true.   This phase contains multiple stories including any communication or publishing of results, IP protection, and a handoff to whomever might be building the final product (which might be the same team).   The final handoff story should also be a spike and the acceptance criteria should include the user stories required for the production deployment.

    BOUNDING AN UNBOUNDED PROJECT

    Now how do you go about making sure research stories don’t go on forever?  How do you bound T and M?  And how do you communicate the cost / value trade offs with management?

    I have found that the previously described framework only works if you apply the following guidelines in conjunction.  Specifically

    1. For any research project to be considered we must have enough information for the project to pass the “sniff test” (i.e. Is it possible in a reasonable amount of time and does it make business sense).
    2. The initial estimate for research projects are based on the expected number of hypothesis iterations and the cost must be inline with the expected project value (i.e. if the research is perceived to have large value it may be worth iterating for a long time).
    3. If the number of hypothesis iterations exceeds the original cost the cost/benefit analysis must be revisited and the project should be canceled if the cost has exceeded expected value.
    CONCLUSION

    What I have presented here is a process by which you can take an unbounded research project and place a structure around it that will work in companies using an agile development methodology.  Besides allowing research projects to function in an agile organization this framework also provides a method for bounding research problems and communicating the cost / benefit trade offs to management and other relevant parties.   For those who have faced similar issues integrating research oriented projects into an agile culture I hope this methodology provides some ideas on how you can better integrate research into your processes.