Management – Speech Wrecko

October 13, 2020October 14, 2020

“If you start me up I’ll never stop …” Until We Successfully Exit

“Hey, our fledgling startup is on path to being the next *INSERT BIG TECH COMPANY NAME HERE* and we think you’re a great fit for our CTO role”. Find me a technical leader who hasn’t been enticed by those words and you’ll have found a liar. So, what happens when one succumbs to the temptation and joins an early-stage startup? Well, if you have been wondering where I’ve been for the past couple of years, I was fighting the good fight at a small, early-stage NLP/Machine Learning based risk intelligence startup. And while I’m not retired or sailing around the world in my new 500-foot yacht, we were able to successfully exit the company with a net positive outcome for all involved. My hope with this post is that I can share some of my acquired wisdom, and perhaps steer the next willing victim down a similar path of success.

If I could sum up my key learnings in a few bullet points, it would boil down to this:

If you don’t believe … don’t join
Be prepared to contribute in any way possible
Find the product and focus on building it
Pick the race you have enough fuel for and win it

What I’d like to do in the rest of this post is break down each one of these items a little further.

If you don’t believe … don’t join

Maybe this goes without saying, but if you don’t believe in the vision, the people, and the product you shouldn’t join the startup approaching you. The CTO title is alluring, and it is easy to fool yourself into taking a job for the wrong reasons. But the startup experience is an emotional slog of ups and downs and it will be nearly impossible to weather the ride if you don’t wake up every day with an unyielding conviction for what you’re doing. As I’ll explain later in this post, you don’t need to believe you’re working for the next Facebook, but you do need to believe you are building a compelling product that has real value for you, your coworkers, your investors, and your customers.

Be prepared to contribute in any way possible

For the first few months on the job I used to go into our tiny office and empty all the trash bins because, if I didn’t, that small office with 5 engineers started to smell! It didn’t take long for someone to call out that I was appropriately titled, CTO (a.k.a. Chief Trash Officer). You might be asking why anybody would take a CTO job to wind up being the corporate custodian, but that is what was needed on some days.

While I have steadfastly maintained my technical chops throughout my career, I hadn’t really written a lick of production code for nearly two decades prior to this job. But with limited resources, it became clear I also needed to contribute to the code base and so I dusted off those deeply buried skills and contributed where I could. When you join a startup with that CTO title, it is easy to convince yourself that you’ll build a huge team, be swimming in resources, and have an opportunity to direct the band versus playing in it. But you’ll quickly find that in the early stages of a startup, the success of the company will depend on your willingness to drop your ego and contribute wherever you can.

Find the product and focus on building it

Great salespeople can sell you the Brooklyn Bridge. And if you’re just lucky enough, you might have a George C. Parker in your ranks. But the problem with great salespeople is they will do almost anything to close the sale and that comes with a real risk that they’ll sell custom work. If that happens over an extended period of time, you will be unable to focus on the core product offering and you’ll quickly find you’re the CTO of a work-for-hire / consulting company.

Startups face real financial pressures that often drive counterproductive behaviors. That often means doing anything necessary to drive growth in revenue, customers, or usage. But as illustrated in the graph below, high product variance will often ultimately lead to stagnant growth.

That’s because with every new feature comes a perpetual support cost. And if you keep building one-off features, and can’t fundraise fast enough, that cost will eventually come at the expense of delivering your true market-wide value proposition. If you allow this to happen, you’ll wind up with a company that generates some amount of revenue or usage but has no real value.

Companies that find true product/market fit should see product variance gradually decrease over time and this should allow the company to grow. Your growth trajectory may be linear when you need it to be exponential, but no per customer feature work will fix that problem and you may need to consider pivoting. If pivoting isn’t an option, it may be time to look for an exit.

As the CTO, a critical part of your job is to help the company find its product/market fit and then relentlessly focus on it. You need to hold the line against distractions and ensure the vast majority of resources are spent on features that align with the core value proposition. If you’ve truly found a product offering that is valued by a given market segment, and you can keep your resources focused on building it, growth will follow.

Pick the race you have enough fuel for and win it

I am an avid runner, and one of the great lessons of long-distance running is, that if you deplete your glycogen store, you’ll be unable to finish the race no matter how hard you trained. In other words, you can’t win the race if you have insufficient fuel. This is also very true of startups. If you’re SpaceX or Magic Leap, you’re running an ultra-marathon and you need a tremendous amount of capital in order to have sufficient time and resources to realize the value. But fundraising is hard, and even if you have an amazing product and top-notch talent, there can be significant barriers to acquiring sufficient capital.

The mistake some startups make is that they continue to run an ultra-marathon when they only have fuel for a 5k and that can lead to a premature or unnecessary failure. If funding becomes an issue, start looking for how your product might offer value to another firm. Start allocating resources towards making the product attractive for an acquisition. Aim to win a smaller race and seek more fuel on the next go around.

Final Thoughts

Taking on a CTO role at an early stage startup can be a great opportunity and lead to enormous success, but before you take the leap make sure you know what you’re getting into. Along the way don’t forget to stop and smell the roses. In the words of fellow Seattle native Macklemore, “Someday soon, your whole life’s gonna change. You’ll miss the magic of these good old days”.

Final Final Thoughts

No startup CTO is successful without support from an army of people. So I’d like to offer some gratitude to the following folks:

Greg Adams, Christ Hurst: Thanks for giving me an opportunity and treating me like a cofounder from day one.
Shane Walker, Cody Jones, Phil LiPari, Pavel Khlustikov, David Ulrich, Julie Bauer, Jason Scott, Carrie Birmingham, Rich Gridlestone, Bill Rick, Zach Pryde, Amy Well, David Debusk, Mikhail Zaydman, Jean-Roux, Bezuidenhout, Sergey Kurilkn (and others I may have forgotten): Thanks for being one the greatest teams I’ve ever worked with.
Brandon Shelton, Linda Fingerle, Wayne Boulais, Armando Pauker, Matt Abrams, Matthew Mills: Thank you for being outstanding board members, mentors, and investors
Ziad Ismail, Pete Christothoulou, Kirby Winfield: Thank you for the career advice during my first venture into the startup world.

*Note: You can read more about Stabilitas, OnSolve, and our acquisition at the links below:
– https://www.onsolve.com/solutions/products/stabilitas/
– https://www.geekwire.com/2020/seattle-based-threat-response-startup-stabilitas-acquired-onsolve/

December 8, 2017

Pair Programming or Bare(ly) Programming

“Sorry we don’t have enough resources, we only have four pairs” – As an engineering leader no other statement has made me cringe more. After all four pairs is a healthy sized team of eight developers.

Throughout my career I have run across CTOs, VPs, directors, development managers, teams, and individual developers who swear by pair programming with near religious devotion. Personally I’ve maintained a healthy dose of skepticism when it comes to pairing as an overarching development philosophy.

As an engineering leader my job is to build products that delight customers in the most efficient way possible. Anecdotally, pairing consistently costs more and hence seems irresponsible to use exclusively as a development technique. But admittedly anecdotal evidence is insufficient so I decided to dig through the research and see if I could find more empirical evidence to support my claim.

Background

Pair programming is an agile software development methodology where two programmers work on the same task using one computer and keyboard. One programmer is called the driver and operates the keyboard and does the primary coding work. The other developer, often called the navigator, is responsible for observing the driver and providing guidance in order to speed up problem solving, improve design, and minimize defects.

The potential negative impact of pair programming is immediately clear to most people. By applying two resources to a task you are effectively doubling the cost. So unless there’s an equal or greater improvement in other project variables, pair programming would be nearly impossible to justify. Exploring the problem through a project management lens, where we have three variables, cost (including resources), time, and quality/scope, If we double our cost we’d expect to see an equivalent decrease in time to deliver or increase in quality or scope (or some factor of each).

In mathematical terms let’s assume the value of any given project X is equal to a weighted linear combination of cost, time and quality/scope.

When pairing our cost is automatically going to double since we’ve applied two resources for a task that in theory can be completed by one.

In order for our project value to remain equal or be better we need our other variables to proportionally change in the right direction. For example if our project now takes 50% less time we could argue we net out even. Or if our scope or quality double, we would similarly be in a good position.

However, In my experience I’ve not seen pair programming live up to these expectations. Instead I’ve seen tasks or user stories take the same amount of time and produce similar results at nearly double the cost. But you shouldn’t take my word for it. Let’s review the literature and see what the experts have to say.

Research

There are actually a fair number of research papers that attempt to prove or disprove the efficacy of pair programming. That said, in my survey of the literature I found most of the research to be ill designed for comparison to real world corporate product development organizations. Specific issues include:

Developer Skills: Most of the studies rely on university students that shouldn’t be compared to seasoned professional developers.

Non Production Environments: The majority of the software used for evaluation is very far removed from real product development environments.

Organization Realities: Finally there is little or no accounting for organizational churn that happens in a real for-profit company

In spite of these issues it’s worth exploring these various research studies and the insights they provide on the impacts of pair programming.

Many of the research papers evaluate the impact of pair programming on effort, which in at least one paper is defined as two times the duration or time required to complete a given task [1]. Specifically, effort increases ranging from 15% all the way to 100% have been observed [2]. In one of the more well conducted studies an effort increase of 84% was seen [1]. Since we know effort is just twice the duration of a single developer we can actually do some math to figure out how much faster pairs complete a task versus a single developer.

Or by using our earlier project management equation, with a little rounding we can assume our pairing time weight would be roughly 9/10 the weight required for a single developer.

This is nowhere near the factor of 1/2 or less we said we needed to make pair programming cost efficient. Well if the research doesn’t support a sufficient decrease in time to completion perhaps there’s research indicating that a given project’s scope or quality will increase enough to offset the difference.

Unfortunately, once again the results are at best inconclusive, but in many cases support an actual decrease in scope and minimal or near zero increase in quality. For example in [2] a reported 29% decrease in productivity was measured for pair programming team when measured as a function of completed use cases.

Regarding quality, even in one of the more optimistic papers we only saw a 10% – 20% increase in quality (measured as test cases passed) [3]. According to [2], we only saw an 8% improvement in quality when measuring actual defects. While these improvements are non trivial, when combined with the time and scope metrics it remains insufficient to offset the associated costs.

Cherry Picking

“But aren’t you just cherry picking the worst examples to justify your case” you might ask? Not really because even in the most optimistic research studies initial results were usually much worse and only improved over time. For example in [3] initial increases in effort dropped from 60% to15% over time. Most of the research attributes these gains in effort to “pair jelling”. In other words, as the pairs get to known each other they become more efficient.

The problem with these studies is that they assume that once a pair jells the gain will hold. However in any real for-profit organization there is potential for high variability in projects and staff which means pair jelling is unlikely to be a one off cost. It is more likely a continuing cost to the business over time.

Several studies also point out that the value of pair programming decreases with simpler tasks [4]. Therefor one must consider the ratio of simple to complex tasks in any given development cycle in order to understand the long term impacts of pair programming. When I evaluated my own teams, I found multiple iterations where 75% of work items where smaller changes that could easily be tackled by a single developer in the same timeframe.

Finally, one paper [5] attempted to justify pair programming by evaluating Net Present Value (NPV). In this paper an argument is made that even if it costs more to pair program, faster time to market warrants the cost. I take issue with this calculation since it does not factor in the opportunity cost of having those extra resources not work on a different higher priority project.

For example if we take the reported 84% increase in effort and assume we finish our project in 9/10 the time of a single developer, we must ask ourselves what happens when a key customer asks for a critical bug fix? I can tell that customer to wait until I finish my current project or I can split my pair and work on both at the same time at the small cost of a 1/10 increase in duration. By splitting my pair I’ve delighted my key customer as quickly as possible at a trivial cost. Clearly you need to factor in the opportunity cost of not delighting that customer when evaluating the value of pair programming.

To Pair or Not to Pair

So should you pair or not pair? There are a lot of reasons a team might use pair programming. In some cases the cost / benefit tradeoff may be worthwhile. Pairing can be very effective at educating new team members, improving the skills of junior team members, cross training, and reducing the cost of complex tasks. If you take anything away from this post let it be:

Challenge the Efficacy of Pair Programming: If your team or engineering manager wants to exclusively use pair programming, don’t blindly accept it. Collect the data to validate if it is really cost effective

Pair when it makes Sense: Use pairing selectively when it makes sense including educating new team members, improving the skills of junior team members, cross training, and reducing the cost of complex tasks.

Factor in Opportunity Costs: Make sure you consider the opportunity costs of projects not being worked on when pairing.

In short don’t allow yourself to be swayed by a dogmatic insistence that pair programming is better. As a leader your job is to challenge your team to delight customers in the most cost effective way possible. Pairing should only be used if it definitively contributes to that cause.

References

[1] Arisholm, Erik, et al. “Evaluating pair programming with respect to system complexity and programmer expertise.” IEEE Transactions on Software Engineering 33.2 (2007). – Summary available at https://pdfs.semanticscholar.org/9787/c9663cad3a1c21550f2e5e365e70fd01d3aa.pdf

[2] Vanhanen, Jari, and Casper Lassenius. “Effects of pair programming at the development team level: an experiment.” Empirical Software Engineering, 2005. 2005 International Symposium on. IEEE, 2005. https://pdfs.semanticscholar.org/40dd/fa666bf367cfffaae421dbd3c6170a3e3dc3.pdf

[3] Cockburn, Alistair, and Laurie Williams. “The costs and benefits of pair programming.” Extreme programming examined (2000): 223-247. http://www.cs.pomona.edu/~markk/cs121.f07/supp/williams_prpgm.pdf

[4] Lui, Kim, and Keith Chan. “When does a pair outperform two individuals?.” Extreme programming and agile processes in software engineering (2003): 1011-1011. ftp://nozdr.ru/biblio/kolxo3/Cs/CsLn/E/Extreme%20Programming%20and%20Agile%20Processes%20in%20Software%20Engineering,%204%20conf.,%20XP%202003(LNCS2675,%20Springer,%202003)(ISBN%203540402152)(479s)_CsLn_.pdf#page=240

[5] Padberg, Frank, and Matthias M. Muller. “Analyzing the cost and benefit of pair programming.” Software Metrics Symposium, 2003. Proceedings. Ninth International. IEEE, 2003. http://wwwipd.ira.uka.de/Tichy/uploads/publikationen/32/metrics03.pdf

October 10, 2017November 13, 2017

Creating a Management Philosophy

Just In case you couldn’t infer this from my previous posts some folks consider me opinionated and occasionally dogmatic. What else would you expect from a born and raised New Yorker, who grew up in a household where arguing your point was considered a cultural birthright!

Unfortunately, while having strong opinions and ideas can be a positive, I’ve found throughout my career that those ideas don’t always resonate with coworkers. Even when those ideas are sound. When I started to lead and manage larger teams this increasingly became a handicap and I soon realized I needed a better way to get my thoughts across.

Enter Pete Carroll. That’s right, the same Pete Carrol who led the Seattle Seahawks to two super bowls and USC to two college national championships. Not too long ago a good friend and neighbor happened to tell me about a management training class he attended based on material from Pete Carroll and his “Win Forever” philosophy. It sounded very compelling and I immediately purchased Pete’s book based on the same concepts.

It was immediately clear from reading this book that Pete Carroll had faced similar challenges earlier in his career.

“But while I had a sense inside me of what we needed, I hadn’t articulated it very well. I didn’t have the details worked out in my own mind so that I could lay them out clearly and convincingly to anybody else”

In short, this book preaches a simple strategy for dealing with an inability to convey your ideas, which is to write them down, iterate on them, and formulate them into a single cohesive vision. By doing so you change the conversation from “hey, here is my opinion” to “hey, I have a strategy for winning and here it is”. Or in Pete’s words;

“by December I finally had a clear, organized template of my core values , my philosophy, and – most importantly – my overarching vision for what I wanted to stand for as a person, a coach, and a competitor”

Armed with a singular vision and philosophy you have a solid foundation to convey your thoughts. And suddenly you have transformed disparate ideas into a recipe for success. The implications of documenting your philosophy are huge and by doing so you will:

Set clear expectations for your employees
Set expectations for executives, higher level managers and peers for how you operate and how it will benefit them
Have a recipe for success that you can continually improve and iterate on

With this information in mind I decided to take years of ideas I had accumulated and started to jot them down. I refined them and wove them into overarching vision. And when I thought about what I was ultimately trying to achieve it became clear that I was always trying to deliver truly innovative software to as many people as possible. And so my Innovate for the Masses™ philosophy was born. I present it to you below unedited. It is a continual work in progress but something that has served me well so far.

	Philosophy: Innovate for the Masses™

	Innovate for the Masses™
	Create unique and defensible value: Products should deliver something truly special that cannot be found in other solutions and simultaneously provide a defensible moat. Build best in class solutions: Products should be fully functional and should not cut corners. We should do everything required to delight customers, nothing more and nothing less. Support all customers: Products should be accessible by all existing and future customers. One off solutions are never okay. Enterprise class reliability and scalability: Product should be robust with 4 9’s reliability and the ability to scale to all customer demands.
	How we do it
	Ruthless prioritization: We question the necessity and value of every feature or piece of code. We only work on things that deliver essential value to the customer. Avoid premature optimization: we only build exactly what the customer needs. No more and no less. Read between the lines: We listen to our customers but don’t just cater to their demands. We find the commonality amongst all our customer’s requirements and build a truly unique and defensible product that surprises, delights, and addresses their needs. Communicate like crazy: We are one team with one vision and one goal. Everybody must constantly be talking to innovate and build cohesive products Right spot right time: We believe every team member plays an important role in the team’s success whether that is in a leading role or a supporting role. Work harder than anyone else: We will win by out working all of our competitors. Don’t chase the competition: We don’t chase every move our competitors make. We pay attention but follow our vision and goals and methodically work towards delivering on them without being distracted.
	Expectations for our people
	Be insanely passionate: Our employees exude passion. We are a passion first organization. Get a lot done / execute like crazy: Our employees are insanely productive. They get more done than anyone else. Care: Our employees give a shit. They care about the product, team, company, and customer like something they hold dear. Have a sense of humor: Our employees laugh. At themselves and each other. We believe you should leave work every day having smiled so much it hurts. Don’t whine or complain: Our employees don’t whine or complain they express their opinions and try to instigate change in the direction they want to see. If a decision doesn’t go their way they disagree and commit. Don’t play politics: Our employees don’t play politics. They lay it all out on the table and do their job to the best of their abilities … that is what they get rewarded for. Dare to disagree: Our employees disagree loudly and proudly. Good disagreement is central to progress. Different opinions are valued and we seek out constructive conflict

Having a vision and philosophy is not all rainbows and unicorns. Creating a philosophy and broadcasting it to your coworkers is the equivalent of driving a giant metal stake deep into the ground. You may find throughout the course of your career that sometimes people don’t agree with your strategy and when they don’t you only have three options, change your strategy, change their minds, or move on. Or again to quote Pete:

“Coach Seifert was specifically adamant that I not change who I was or my mentality. He said clearly “Pete, you’ve got to do it the way you know how.” After my experience in New York, I wondered if I shouldn’t try to be more political, but the advice I got from the two mentors was uncompromising – and some of the best I ever received.”

In closing if you are anything like me or Pete Carroll I strongly encourage you to write down your great ideas and formulate them into a cohesive philosophy. It will be will worth your while.

**For reference Pete Carroll explicitly calls out John Wooden for influencing his strategies and techniques. I highly encourage people to also read John Wooden’s book “Wooden on Leadership”

September 18, 2017October 8, 2017

Managing Research Projects in an Agile Development Environment

Anyone who has worked in an agile organization has found that certain projects don’t quiet fit the agile mold. Nowhere is this more apparent than with research oriented projects. After all if there is complete uncertainty in the scope and outcome of a project, as would be the case in a research project, how do you create user stories and estimate story points? And if you can’t create stories and estimate the associated costs how can you hold your team accountable, communicate status to the rest of the organization, and make cost / benefit tradeoffs? Simple! You can’t.

I’ve personally dealt with this issue after hiring several researchers to work on an agile software product team. Initially, I struggled to interleave our research projects with our other production work so I started looking for a solution. The answer to my problem came after reviewing the agile literature and the scientific method and concluding that research projects really just represent an extreme of what the agile process is ultimately trying to solve. Below I will walk you through how I arrived at this solution and details on how you can apply similar tactics in your own research organization.

AGILE PROCESS

Early in my career at Microsoft someone handed me a copy of Steven McConnell’s book Code Complete.

At the time my greatest take away from that book was the concept of the “Cone of Uncertainty”. The “Cone of Uncertainty” states that the uncertainty of a given project decreases as time progresses and more details are flushed out.

Historically the “Cone of Uncertainty” was dealt with by creating detailed upfront plans and using waterfall project management approaches. The trouble with those methodologies is that they’re extremely resistant to scope change. Largely because scope change reintroduces uncertainty.

The agile manifesto attempts to eliminate the “cone of uncertainty” problem by following the principle of “Responding to change over following a plan”. Most agile methodologies use some form of iterative development to reduce uncertainty, with the idea being that if you’re working on smaller well defined chunks of a larger project uncertainty is removed and the project can slowly adapt to changing requirements. Mike Cohn wrote in an article titled “The Certainty of Uncertainty”.

“The best way to deal with uncertainty is to iterate. To reduce uncertainty about what the product should be, work in short iterations and show (or, ideally give) working software to users every few weeks. Uncertainty about how to develop the product is similarly reduced by iterating. For example, missing tasks can be added to plans, inadequate designs can be corrected sooner rather than later, bad estimates can be amended, and so on.”

If I take the above information together I can conclude two things. First, the agile method attempts to reduce or eliminate uncertainty by making every project a function of smaller work items iterated over time. Or framed in mathematical notation:

Where: T = Max Iterations, M = Backlog, N = User Stories belong to M

Secondly, if a research project is really just a project with maximum uncertainty then the same framework should apply. Only there would be an unbounded number of work items over an unbounded amount of time. Or framed in mathematical notation:

According to this logic a research project should actually work within an agile framework. We just need to figure out how to construct M (i.e. backlog) and how to bound M and T (i.e. number of iterations).

SCIENTIFIC METHOD

So what are reasonable user stories for a research project and why are they potentially infinite? It occurred to me that research in general follows the scientific method and that the scientific method may be a good framework for story generation.

In essence the scientific method can be boiled down to three phases: a research phase, an iterative hypothesis testing phase, and a communicate or productize phase. The unbounded component of research is that many hypotheses end in failure leading to another hypothesis that must be tested and this can potentially go on ad nauseam. This provided me a compelling framework for how to break research into user stories.

The first story in any research project correlates to the first phase in the scientific method. This story should be a time bounded spike that frames the initial question, covers any background research, and has an acceptance criteria of generating the required stories for the next phase of the project, hypothesis testing.

The next set of stories are all part of the hypothesis testing phase. These stories include any development work required to test the hypothesis, any data collection required, running the tests, and analyzing the results. If the hypothesis proves false the team should circle back to the background research phase and continue on with the process.

The final phase in this framework is only relevant when a hypothesis is proven to be true. This phase contains multiple stories including any communication or publishing of results, IP protection, and a handoff to whomever might be building the final product (which might be the same team). The final handoff story should also be a spike and the acceptance criteria should include the user stories required for the production deployment.

BOUNDING AN UNBOUNDED PROJECT

Now how do you go about making sure research stories don’t go on forever? How do you bound T and M? And how do you communicate the cost / value trade offs with management?

I have found that the previously described framework only works if you apply the following guidelines in conjunction. Specifically

For any research project to be considered we must have enough information for the project to pass the “sniff test” (i.e. Is it possible in a reasonable amount of time and does it make business sense).
The initial estimate for research projects are based on the expected number of hypothesis iterations and the cost must be inline with the expected project value (i.e. if the research is perceived to have large value it may be worth iterating for a long time).
If the number of hypothesis iterations exceeds the original cost the cost/benefit analysis must be revisited and the project should be canceled if the cost has exceeded expected value.

CONCLUSION

What I have presented here is a process by which you can take an unbounded research project and place a structure around it that will work in companies using an agile development methodology. Besides allowing research projects to function in an agile organization this framework also provides a method for bounding research problems and communicating the cost / benefit trade offs to management and other relevant parties. For those who have faced similar issues integrating research oriented projects into an agile culture I hope this methodology provides some ideas on how you can better integrate research into your processes.