Can You Afford to Experiment?

Lean Startup & Minimum Viable Products have replaced planning and are helping entrepreneurs find product-market fit more quickly. But these tools also have costs and risks.

Historically, entrepreneurs started building companies by undertaking long, detailed planning efforts, in order to carefully understand the market; identify customers, competitors, and suppliers; and forecast the financial performance of the nascent business. Practically, this often translated into “build-it-and-they-will-come” or “waterfall” development processes, as discussed by Eisenmann & al (2013).

While research shows that this practice does have positive effects (Delmar & Shane, 2003), business planning has been criticized by many. As discussed by Steve Blank (2013), a challenge is that the focus on business planning often implies postponing contact with the market—and, critically, receiving feedback from the market—to very late in the development process. And as several case studies suggest (for example, the story of Boo.com), this is sometimes too late. Much of the rationale of this approach was that building products was expensive and time-consuming, so it was helpful to try and calculate all risks in advance.

But now building products is cheaper, thanks to the emergence of a variety of new technologies, such as 3D printing, crowdsourcing, and cloud computing. So, today a common route is to build an early version of a product—or, a minimum viable product (MVP)—and get customer feedback before making substantial investments. Consider the famous example of Dropbox. In testing what was then a radically innovative idea, the Dropbox CEO simply created a video demonstrating the features of the product he was building. He received positive feedback, continued developing the product, and we know the rest of the story. While there are numerous ways to build a MVP, the basic idea is about an extremely simple product that “summarizes” the value the customer would get from the final product. By showing the MVP to potential customers, the team learns more about the market and has the opportunity to iteratively build something close to what customers want. This notion became popular thanks to the Lean Startup.

The 'Experimental Startup'

Popularized by the work of Eric Ries (2011, 2017), the Lean Startup has largely become the “language” of modern entrepreneurship. The name “Lean” comes from Toyota’s Lean Management but is perhaps somewhat misleading: Ries’ core message is that entrepreneurs need to learn about the market by running experiments, so we could think of it as the “Experimental Startup.” Here is how the process looks: you come up with a product idea, translate that idea into a set of hypotheses, build the MVP to test the most critical of those hypotheses, update them based on the feedback you receive, and move on to your next MVP. Moving to the next MVP might involve merely optimizing the previous version. Or it could require a more fundamental “pivot” to something markedly different. Once you stop getting informative feedback—or, when most of your hypotheses appear to hold—you have found what some call “product-market fit,” and you can start thinking about scaling the business.

The key advantage of this approach is the possibility of learning about what to build before actually building (most of) it. This might potentially diminish the risk of starting a company. As Reid Hoffman (2012) puts it, “Whatever the situation, actions, not plans, help you test your hypotheses against reality.” However, while it is hard to not recognize the innovative contribution of this approach, one-size-fits-all recipes rarely work. As the Lean Startup literature also discusses, there are sectors—from pharmaceutical to finance, from automotive to energy—where running an experiment is prohibitively costly, due to the capital needed to build the MVP or to high regulatory barriers. And even when building MVPs is feasible, there may be “strategic costs” to experimentation.

Strategic Costs to Experimentation

Perhaps the most intuitive challenge of taking the experimental approach is idea theft. Especially in the era of massive tech giants, it is well-known that large companies keep an eye on startups to identify and capture promising ideas. Bringing the MVP to market early on might expose the idea during a phase when the venture does not have substantial protection from  imitation, such as a strong brand or network effects. Intellectual property protection might mitigate this risk, but is unlikely to be a complete defense, especially in sectors such as software, where technological change is particularly rapid.

Bringing a very preliminary MVP to market might also affect your reputation. For rookie entrepreneurs, as opposed to serial entrepreneurs with a strong “resume,” building a strong reputation is critical. Particularly in settings in which feedback is particularly “sticky”—for example, when there is an established rating system that lets users share their views on the product—it might make sense to pay attention to the quality of your first MVP. Strategies to mitigate reputational risks, such as using a different brand or running relatively “private” experiments, might be only partially effective.

In addition, it may be costly to keep iterating towards a product that is closer to what customers want. In software, people talk about technical debt (MacCormack & Sturtevant, 2016): the notion that, if you build something with “poor” foundations, fixing and improving it later on might be difficult. This cost is often larger in non-pure-play-software products, such as wearables or electronics. Plus, changing the product sometimes requires changing the team, and that may create additional challenges. One way to possibly diminish such costs of adaptation is to design a modular product structure (Pil & Cohen, 2006). But research suggests that this approach also has downsides, including the risk of idea theft (Ehiraj & al, 2008).

Finally, and somewhat more subtly, experimentation might not always lead to “good learning.” As a lot of research shows, organizations face several challenges to learning. In the startup setting, effectively interpreting market feedback may be an arduous task. Sometimes entrepreneurs undervalue the signals they get from their experiments, and do not pivot enough. Other times, they overvalue them, and update their views in inaccurate ways. These and other biases make it challenging to learn effectively from experiments.

Peter Thiel (2014) writes that “Darwinism may be a fine theory in other contexts, but in startups, intelligent design works best.” While there is little doubt that experimentation is an important tool for new ventures, it may not always be the optimal strategy. As much of the strategic management research suggests, contingencies matter. Therefore, not unlike other important strategic situations, entrepreneurs must examine their internal strengths and the environment carefully before taking the experimental route.

LEARN MORE

Blank, S., 2013. Why the Lean Startup Changes Everything. Harvard Business Review, 91(5), pp.63-72.

Contigiani, A. and Levinthal, D.A., 2019. Situating the Construct of Lean Start-up: Adjacent Conversations and Possible Future Directions. Industrial and Corporate Change, 28(3), pp.551-564.

Delmar, F. and Shane, S., 2003. Does Business Planning Facilitate the Development of New Ventures? Strategic Management Journal, 24(12), pp.1165-1185.

Ethiraj, S.K., Levinthal, D. and Roy, R.R., 2008. The Dual Role of Modularity: Innovation and Imitation. Management Science, 54(5), pp.939-955.

Eisenmann, Thomas, Eric Ries, and Sarah Dillard. Hypothesis Driven Entrepreneurship: The Lean Startup. Harvard Business School Background Note 812-095, December 2011. (Revised July 2013.)

MacCormack, A. and Sturtevant, D.J., 2016. Technical Debt and System Architecture: The Impact of Coupling on Defect-Related Activity. Journal of Systems and Software, 120, pp.170-182.

Hoffman, Reid and Casnocha. Ben, 2012. The Start-up of You: Adapt to the Future, Invest in Yourself, and Transform your Career. Random House.

Ries, E., 2011. The Lean Startup. New York: Crown Business.

Ries, E., 2017. The Startup Way: How Modern Companies Use Entrepreneurial Management to Transform Culture and Drive Long-term Growth. Currency.

Pil, F.K. and Cohen, S.K., 2006. Modularity: Implications for Imitation, Innovation, and Sustained Advantage. Academy of Management Review, 31(4), pp.995-1011.

Thiel, P.A. and Masters, B., 2014. Zero to One: Notes on Startups, or How to Build the Future. Currency.