The Theory of Disruptive Innovation: Science or Allegory?
“Every age has a theory about the past and the present, of what was and what is, a notion of time: a theory of history…Our era has disruption…[1]”
– Jill Lepore, David Woods Kemper ’41 Professor of American History at Harvard University
Clayton M. Christensen’s Theory of Disruptive Innovation is now more than 20 years old, but it still holds a prominent place in business strategy. His first book, the “Innovator’s Dilemma” is one of the best-selling business books of all time. The idea of disruptive innovation has become so common that it is now used to explain nearly every existent or threatened disturbance - everything from the success of Tesla Corporation[2] to the workings of President Trump’s cabinet[3]. In a critique in The New Yorker, Harvard University Professor of History Jill Lepore argues that the theory has become a kind of secular religion, useful predominantly for soothing our “profound anxiety about financial collapse” by "providing an atavistic explanation for the rise and fall of great firms.” In response, Professor Christensen angrily disagreed, but he too contends that the theory has been overused. In an interview with the Harvard Business Review, he said, “I never thought … that the word disruption has so many connotations in the English language, that people would then flexibly take an idea, twist it, and use it to justify whatever they wanted to do in the first place.[4]”
Despite its common misapplication, The Theory of Disruptive Innovation is also an interesting scientific conjecture, and an extension of previous thinking on Joseph Schumpeter’s process of creative destruction in industries. It breaks from the previous literature by focusing blame for displacement of industry leaders on management rather than on faltering firm capabilities. Christensen’s theory attempts to provide an explanation for why firms, with abundant remaining capabilities, might still fail in response to new competition.
Christensen and his coauthors have never conducted a quantitative study to test the predictive power of the theory[5], but several other authors have done so. Several studies investigated growth and failure patterns in various industries, and were unable to validate the model[6]. A few scholars have evaluated the home industry for the theory – computer hard disk drives – and found evidence contradictory to its basic predictions[7]. In response to these failures of empirical confirmations, Christensen blamed over-simplified abstractions of the model and its measures. Single failures of prediction, like Christensen’s own use of the theory to disparage the future of the iPhone, have been dismissed as isolated incongruities or resolved post-hoc. For example, Christensen now says the iPhone is disruptive after all, but to PCs rather than other phones.[8]
In his defense of the theory, Christensen has argued repeatedly that it can only be tested by comparing it to numerous historical cases.[9] Bajjir Baatartogtokh and I decided to follow his advice by investigating each of the 77 examples listed in his first two books. This non-random sample has the disadvantage of being unrepresentative and advantaging a supportive finding. The Chronicle of Higher Education later described our work as favoring the theory by evaluating the theory “on its own terms.”
To evaluate these 77 cases, we interviewed 82 experts with knowledge of one or more of the cases. “Fifty-eight percent of the surveyed respondents were academics; 18% were nonacademic authors of book-length historical analyses; 10% were financial analysts of the industries involved; and 14% were participants in the industries.10" To improve the objectivity of our study, we began each interview with a structured survey about the theory’s three main premises and its conclusion[11].
1) Incumbents are improving along a trajectory of innovation.
2) The pace of sustaining innovation overshoots customer needs – opening the door to a disruptive innovator.
3) Incumbents have the capability to respond but fail to exploit it.
4) Result: Incumbents flounder as a result of the disruption.
We checked our understanding of these claims with various experts on the theory. Our biggest challenge was in developing a measure for a firm being “disrupted.” Did that mean the firm failed, lost market share, stopped being an industry leader, or something else? Eventually, we settled on “floundered – meaning the loss of substantial market share.” To check the 77 cases for the existence of anomalies, we surveyed and interviewed historical experts on each case.
In 2015, we published our results in the Sloan Management Review. Our case experts disagreed with Christensen’s interpretation of the large majority of the cases. For about 30% of the cases, industry leaders were not on a trajectory of sustaining innovation (e.g. 19th century beef processing). For nearly 80%, sustaining innovation never overshot what mainstream customers wanted (e.g. airlines and surgery). For almost 40% of the cases, incumbent firms did not have the capabilities to respond to the new threat (e.g. postal services responding to email)[12]. And in almost 40% of the cases, incumbents were not displaced by a disruptive entrant. According to the experts we interviewed, only seven of the 77 cases—or 9%—matched the theory’s premises and prediction.
We concluded that the Disruptive Innovation, as described by Christensen, does sometimes occur. The seven matching cases show this conclusively. We also concluded that the conditions for using the theory are rare, and its predictive power is limited. We concluded that the burden of legacy commitments (the steel industry), changing scale economies (meat packing), and the laws of probability (e-commerce), better explained the troubles faced by the firms featured in the seven cases.
Suggestions for Managers
Competition from new, potentially disruptive technology is a common element of today’s business world. Christensen suggests that managers always should contend for market share with new entrants. If these new contenders possess technology that is unfamiliar or unsuited to existing business units, then incumbents should create new semi-autonomous organizations to act as the competitor. Unfortunately, this advice is not reliable because it ignores the strategic constraints—and opportunities—created by capability differences. It also ignores any analysis of the costs and benefits of winning in the transformed market space.
The story of Eastman Kodak was not one of the 77 cases we considered, but it has often been referenced by Professor Christensen and it provides a useful example of the dangers of excessive application of the theory of Disruptive Innovation. Christensen himself worked with Kodak on its plan for responding to disruption by digital imaging, and he praised their implementation[13]. For a time, the company was the largest maker of digital cameras in North America—holding 28% of the market. Its main competitor in digital imaging, Fujifilm, took a different route. Though it too developed some digital cameras, it also moved aggressively to re-purpose its capabilities in chemical technologies for use in other markets where high-quality thin films were required (e.g. coatings for flat panel displays). Kodak was forced into Chapter 11, while Fuji is a $21 billion business. Of this, digital cameras is about five percent.
The former Kodak executive who consulted with Christensen, Willy Shih, now argues that Kodak made several serious errors[14]. First, Kodak had enormous technological advantages in film production and none in digital sensors or electronics. Like Fuji, Kodak should have re-purposed these capabilities. Second, Kodak should have thought more deeply about the value of winning in the digital imaging market. With no regular sales of consumable products—such as film—digital imaging is a small business with narrow margins[15]. Third, Kodak should have recognized the constraints imposed by their legacy obligations. These meant that the firm could not survive as a small digital business.
In our discussion with other industry experts, we heard similar suggestions about how managers should respond to a potential “disruptive” competitor. As we write in our article, there suggestions were most common:
Leverage existing capabilities. “Managers should analyze how their existing capabilities can be deployed most profitably. If current capabilities can be used or extended, it may make sense to expand into a new market.”
Calculate the value of winning. “Christensen and his collaborators seem to assume that no matter what industry or market a company is in, it should fight to maintain control. But this is folly. In many of the 77 cases, factors such as low barriers to entry, the emergence of substitutes, and an increase in the number and aggressiveness of rivals turned once-profitable industries into profitless deserts.”
Consider working collaboratively. “The prospect of an entrepreneur with new technology potentially disrupting incumbent businesses can make managers wary of cooperating with entrants. In several of the cases we explored, however, incumbents recognized the potential for working with new entrants.”
For entrepreneurs, our results may have two implications. First, incumbent firms will seldom ignore new entrants – even if their products match the attributes of a disruptive entry. Second, the possession of a potentially disruptive technology or business model does not imply business success. Most markets will remain sharply contested by both incumbents and new firms. Sustainable profits still require developing a strong comparative advantage.
Conclusion
The Innovator's Dilemma and its successor, The Innovator’s Solution, have had a great impact on business leaders worldwide. Readers should be aware that many of the cases discussed in the book do not match the cause and effect theory that drives the managerial insights of the book and that systematic tests of the theory have demonstrated that the process described by Christensen occurs only rarely. We believe that the book provides a defensible warning about a potential mechanism for competitive displacement, but its application is appropriate in only very limited settings, and its recommendations are postulations without scientific basis. We believe that managers should focus their strategic planning on how best to use firm capabilities.
Editor’s Perspective
Jon Eckhardt
The Innovator’s Dilemma is an incredibly influential book. Many entrepreneurs and innovators use insights from the book to develop business strategy, and I have seen many investors use the book as a guide to developing their investment thesis. Andrew King and Bajjir Baatartogtokh have done a great service by taking the time to carefully examine how the data in the book aligns with the theory that was presented. King’s essay makes the case that the theory’s uncertain constructs may encourage misapplication and a resulting waste of time and money. If you read The Innovator’s Dilemma, you should also read King and Battartogtokh’s “How Useful is the Theory of Disruptive Innovation?”
References
[1] J. Lepore, “The Disruption Machine: What the Gospel of Innovation Gets Wrong,” New Yorker, June 23, 2014
[2] https://www.vox.com/the-big-idea/2017/6/26/15872468/tesla-gm-ford-valuation-justifying-disruption
[3] http://fortune.com/2017/07/27/scaramucci-trump-disruption-startup/
[4] “Clay Christensen on the Recent Debate Surrounding His Theory of Disruptive Innovation,” interviewed by Adi Ignatius, June 27, 2014, www.youtube.com.
[5] The closest they have come is a study of the match existing capabilities, proposed products, and success within Intel. This might support the idea that firms are better at sustaining innovation.
[6] Danneels, Erwin. "Dialogue on the effects of disruptive technology on firms and industries." Journal of Product Innovation Management 23.1 (2006): 2-4.
[7] King, Andrew A., and Christopher L. Tucci. "Incumbent entry into new market niches: The role of experience and managerial choice in the creation of dynamic capabilities." Management science 48.2 (2002): 171-186. McKendrick, David, Richard F. Doner, and Stephan Haggard. From Silicon Valley to Singapore: Location and competitive advantage in the hard disk drive industry. Stanford Business Books, 2000.
[9] Christensen, Clayton M. "The ongoing process of building a theory of disruption." Journal of Product innovation management 23.1 (2006): 39-55.
[10] King, Andrew A., and Baljir Baatartogtokh. "How useful is the theory of disruptive innovation?" MIT Sloan Management Review 57.1 (2015): page 79.
[11] These are clearly outlined in Chapter 2 of Christensen and Raynor.
[12] Willy Shih has pointed out this is often the case in the transition from analog to digital technology.
[13] "Managing disruption: an interview with Clayton Christensen: Clayton Christensen talks with James Euchner about how companies can anticipate disruption and the challenges of managing disruptive change.." The Free Library. 2011 Industrial Research Institute Inc. 04 Aug. 2017
[14] http://sloanreview.mit.edu/article/the-real-lessons-from-kodaks-decline/
[15] Reflecting on the demise of Fuji’s long time rival, Fuji’s CEO summed up his view: “Kodak aimed to be a digital company, but that is a small business and not enough to support a big company.”
Additional search terms: disruptive change, disruptive new companies, black swans, obsolete businesses
