The product innovation process has traditionally been undertaken using a gated model, consisting of a series of defined stages and gates or Go/Kill decisions points. Such Stage-Gate® processes were developed to deal with the random and disorganized, often chaotic, approach to new-product development, once prevalent in many major firms (and which is still a challenge for too many small and medium-sized companies). A typical gated model is shown in Figure 1.
A structured gating process for product innovation has clear benefits: Such a system allows management to select the best ideas and projects with more insight and knowledge; it also reduces the risk and costs of project failure and increases the chance of new products success.
An exciting new development in the field of innovation management combines the principles and methods of Agile project management – whose origins lie in the software development world – with the tried-and-proven Stage-Gate® method for the development of new services and physical products. Recent evidence shows that this new hybrid approach speeds up the innovation process, makes it more flexible and iterative, and ensures a measurable output early in the process.
A solid and proven model for Agile innovation for manufactured products does not exist yet (such as the traditional Stage-Gate® model). Early-adopters’ experiences, however, reveal how to practice this hybrid model for new physical product development, and also prove that it works. Such early experiences also identify some key challenges and inconsistencies that one must be aware of before implementation.
Stage-Gate® is a widely-used model for product development and innovation: An estimated 75 percent of major U.S. companies have adopted Stage-Gate®, and in Europe, most major manufacturers use a version of Stage-Gate®.¹ The benefits of Stage-Gate® can be summarized as follows.² Stage-Gate® is:
Stage-Gate® thus remains a highly relevant model for managing product and service innovation.
In the last decade, many companies have made their Stage-Gate® system leaner by analyzing the value-stream, simplifying the process, streamlining deliverables, and removing non-value-added activities. In most cases, a more efficient and leaner innovation process has been the result. “Lean” has not solved all the challenges, however: Issues remain, for example, dealing with uncertainties and ambiguity; coping with rapidly changing customer needs and wants; and accommodating the realities of a faster-paced world where plans often change. Experience have demanded that Stage-Gate® become more adaptive, more agile, and more accelerated.
A new approach – namely integrating principles and methods from Agile project management into Stage-Gate – promises to deal with these challenges and appears to yield dramatic performance results. As noted in a recent Research-Technology Management article: “Indeed, integrating Agile-Scrum methods into Stage-Gate to yield this new Agile–Stage-Gate hybrid model may be the most exciting and significant change to the new-product process since the introduction of gating systems more than 30 years ago.”⁵
Why? First, because this novel Agile-Stage-Gate® hybrid approach provides a framework for dealing with uncertainties and ambiguity in the front-end, accelerates the process through the use of time-boxed iterations, and focuses on the results via development of tangible product increments as the measure of progress.
…the new hybrid method can be used in all stages from Discovery (ideation) to Launch.
Second, the new method also has a major potential for increasing success rates of new products: the model requires that the project team interact with users and customers, starting in very early stages, thus getting valuable feedback and early market validation.
Finally, Agile-Stage-Gate is not “rocket science”. Rather, Agile-Scrum is fairly straightforward to employ as a working method as part of the stages of Stage-Gate®. Moreover, although usually initially deployed in the two technical stages, namely Testing and Development in Figure 1, the new hybrid method can be used in all stages from Discovery (ideation) to Launch, as shown in Figure 3.⁶
Agile project management is based on principles from The Agile Manifesto (see Figure 4).⁷ Agile was developed in reaction to major failures using the dominant “waterfall” way of managing large IT projects. Waterfall focused on the big, long-term goal: the final product and its major features. But requirements tend to change rapidly in IT projects; the features and criteria defined when the project was initially planned often were no longer valid by the end of a 12- to 18-month development cycle. Committing early to features and schedule means that compromises will be needed late in the game; early commitments to large features, long schedules, long feedback loops, and the replanning inherent in traditional product development processes create inefficiencies and slow the development cycle.⁸
In Agile, development projects are based on the principles of self-organizing teams; close dialogue with users and customers; many short iterations with functional and visible results; interdisciplinary collaboration between business and development; and adoption of changes as the project progresses.
The results in the IT world have been very positive, in particular in reducing development time.⁹ Agile project management has therefore gained ground as the new dominant development methodology in many IT projects and IT-driven companies in the last five years. Figure 5 shows results achieved with Agile from an HP survey of over 600 IT product developers across multiple businesses.¹⁰ Nearly 50 percent of the IT professionals in the survey reported increased customer satisfaction and more than 40 percent witnessed reduced development costs and shorter times to market.
Experience shows that certain IT projects are more suited for Agile than others. For example, the largest bank in Denmark, Danske Bank, reports that more complex projects will gain more from Agile than simpler projects which can be well-defined upfront.¹¹ Danske Bank’s experience is that successful Agile projects need a clear project owner, and also project team members that are experienced in Agile development. Developers were trained initially and then helped by an Agile coach for two months during implementation. Performance results were significant at Danske Bank, with development times cut in half by using Agile. Currently the bank has more than 40 development teams using Agile project management, in parallel with the classic waterfall model, each model being used for different purposes and in different contexts.
Scrum is one of the more popular versions of Agile, and is indeed the version that is most often used in conjunction with Stage-Gate. Scrum was first identified in 1986 as “a flexible, holistic product development strategy where a development team works as a unit to reach a common goal” as opposed to a “traditional, sequential approach”.¹² The basic framework of Scrum consists of:¹³
The elements of the Scrum process are illustrated in Figure 6.
There are three major roles in the Agile-Scrum framework, which together are called “The Scrum Team”:
Most notably is the absence of the traditional project manager role. Traditionally, the project manager’s role is to manage the project in terms of scope, cost and time using a project plan with defined deliverables, detailed budgets, a milestone plan; to facilitate project meetings, ensuring progress, communication; and sometimes to be involved in project staffing.
The project management role is divided somewhat differently in an Agile-Scrum team:
A handful of leading manufacturers have begun to experiment with combining their traditional Stage-Gate process with the newer Agile methods. The resulting Agile-Stage-Gate® hybrid model is still new, but initial experiences from lead users are very positive. This hybrid version of Agile (such as Scrum) and Stage-Gate® seems to be a promising new innovation model.¹⁵
The benefits of combining Agile with Stage-Gate®, based on a number of cases from Denmark, Sweden, Germany and the U.S., can be summarized in five areas, listed in Table 1.¹⁶
The most provocative results flow from two factors in Table 1:
How much Agile-Stage-Gate can increase success rates and accelerate development time is still not yet fully documented, and may vary across industries and types of organizations. But in one Danish study of five manufacturing firms, the results were extremely positive on a number of performance metrics: more adaptive (faster response to changing product requirements), better team communication, higher team morale, and most important, faster to market.¹⁷
Agile-Scrum provides a simple methodology that can be readily built into Stage-Gate® to speed up the innovation process, make it more flexible and iterative, as well as ensuring user-feedback and a measurable output during the process. By working in the sprint format, speed and fixed-time are emphasized: the tasks may vary, but not the length – a sprint cannot last longer than what was agreed by the team. This applies to all stages in the innovation process.
The greater flexibility in content inherent in Agile-Stage-Gate is well-suited to the innovation process, where much uncertainty and ambiguity may initially exist.
The greater flexibility in content inherent in Agile-Stage-Gate is well-suited to the innovation process, where much uncertainty and ambiguity may initially exist. That is, in some development projects, no matter how much voice-of-customer work or technical assessment is done in the early stages, the product definition is still not totally certain as development begins. In Agile-Stage-Gate, however, the product is not defined in detail in advance, but rather its definition is determined as part of the development process and as the team moves forward. Note that Agile-Stage-Gate® is an iterative process which builds in a user focus throughout the process: the project team builds product increments to test and learn from customers and users along the way at every stage.
Developing a physical product or service is obviously quite different than developing software, and so the notion of “product increments” created in each sprint in an Agile-Stage-Gate system must be modified somewhat from the IT-version of Agile. But early-adopter manufacturing firms have made the adjustment, and typically define “product increments” as “any deliverable that is tangible, the result of work done by the project team”. These tangible deliverables can be, for example, the results of market research (voice-of-customer work); computer generated 3D drawings of the product concept; detailed design drawings; protocepts (a product version between a concept and a ready-to-trial prototype); rapid prototypes; and early working models – anything that can be demonstrated to stakeholders, including customers, for feedback.
Agile-Stage-Gate® is used most often during the technical stages – Development and Testing – but also applies to the early or “fuzzy front end” stages of the innovation process, namely Ideation (discovery), Concept Development, and Building the Business Case. Figure 7 illustrates how Agile principles and Scrum can be built into these earlier stages leading to Development in a Stage-Gate® framework.
The iterative (spiral) sprints in the early stages include the Idea Sprint and Concept Sprint, followed by a number of Development sprints. Each of these early stages typically are made up of several sprints before entering the gates in Figure 7. The sprint reviews, which follow each sprint, are in effect “mini-gates” to evaluate each increment done in the individual sprint; by contrast, each gate in Figure 7 is a more in-depth evaluation of all the increments completed in the project thus far, as well as a Go/Kill or investment decision point.
Each of the multiple sprints in the Idea and Concept stages lasts 1-4 weeks, and consists of a number of repeated actions within the Scrum framework. Early-stage sprints and detailed actions are illustrated in Figure 8, and while they may be executed in different ways, some important caveats and best practices exist, and are highlighted below:
The Danish company, Danfoss, undertook a major waste-water treatment innovation project, whereby Danfoss designers and developers co-ideated new solutions and created scenarios jointly with users. Both groups worked together at the user’s waste-treatment plant: Here they followed the users’ treatment process step-by-step, and imagined how a new digital monitoring solution might help users get their jobs done better. (This case is documented by Buur and Matthews, who label the method “Participatory Innovation” to emphasize the participation of users early in the process¹⁹). Developing user-stories with this approach is ideally a part of the second idea sprint – a test of early ideas – and thus guides subsequent concept sprints.
The Sprint Planning Meeting is important to plan the sprint and define “Done” (referred to as DoD = Definition-of-Done). In the world of physical products, “Done” may be obvious at later sprints, when development is underway; less evident is “Done” in the earlier stages, where the product is undefined and nothing physical is yet available. Here are some specific guides for the early stages:
An important step following each sprint is to revisit the backlog. For example, the Idea Backlog is refined after each idea sprint. This Idea Backlog ultimately becomes the Concept Backlog once it passes the first gate in Figure 7. Similarly, the Concept Backlog is updated after each successive concept sprint. The ongoing refinement of the product backlog must be transparent and visible to all project teams and team members. The Product Owner is manager of the Idea and Concept Backlog and decides what should be prioritized and included in the backlogs (for example, what product features), based on an understanding of what creates value to the customer and to the business.
Best practice visual, user-oriented and collaborative methods that are utilized within Agile-Stage-Gate® include:
Combining Agile project methods, such as Scrum, with Stage-Gate® has the potential to yield more productive and faster product and service innovation. Agile-Stage-Gate® gets the product right via multiple, early customer-validations; it deals with uncertainty and ambiguity regarding what the product solution might be; it focuses project teams on the one project, thus accelerating projects; and it fosters much stronger team morale. Examples of early adopters of this new system are appearing around the world where companies successfully bring these two models together – a hybrid approach which delivers the best of each model – for innovation for both digital solutions, physical products and service products.
These are early days however. And more evidence and experience-based results are required – results evaluated in terms of objective measures such as development times-to-market and new-product success-rates. Moreover, there remain a number of inconsistencies and paradoxes that early adopters are still working out, especially in the world of manufactured products, and where outside expert help might be needed.²¹ Nonetheless, early results are very promising, sufficiently so that any firm seeking improved product development performance should investigate and consider these new Agile-Stage-Gate hybrid methods.
Are you interested in learning more about Agile-Stage-Gate®? Please feel free to contact GEMBA Innovation.
By Tomas Vedsmand, Søren Kielgast & Robert G. Cooper
Tomas Vedsmand is an innovation professional with more than 15 years of experience with innovation, business development and strategy. He has worked on structuring innovation processes as well as managing innovation in numerous consultancies, start-ups and corporations. His focus in recent years has been combining the Stage-Gate with agile project principles such as iterations, user-focus and self-organizing teams. Tomas has experience in sectors such as food, retail, construction, clean-tech and the service sector. He has a PhD in food innovation and is partner in GEMBA Innovation and 8ideas.
Søren Kielgast, Partner, Gemba Innovation has more than 20 years’ experience in innovation, business development and management from a number of international technology companies and from hands-on efforts related to the establishment of Teknologisk Innovation, a government funded seed-incubator, which has nurtured more than 100 startups in six years. Søren has experience in developing IT based innovation management systems and is a certified instructor in the team learning and meeting system Zing. Søren has implemented Zing in a number of companies and organizations in Scandinavia and Germany.
Dr. Robert G. Cooper is one of the most influential innovation thought leaders in the business world today. He pioneered the original research that led to many groundbreaking discoveries including the Stage-Gate® Idea-to-Launch process. He has spent more than 30 years studying the practices and pitfalls of 3,000+ new product projects in thousands of companies and has assembled the world’s most comprehensive research on the topic. He is President of Product Development Institute Inc.; ISBM Distinguished Research Fellow at Penn State University’s Smeal College of Business Administration, USA; and Professor Emeritus, DeGroote School of Business, McMaster University, Canada. Cooper has published over 120 articles in leading journals on new product management and has also written eleven books on new product management, including the best-seller, “Winning at New Products 4th ed” which has become the ‘bible’ for new product development.
2. About Stage-Gate®: see “The Stage-Gate system: A roadmap…” at www.bobcooper.ca/articles/next-generation-stage-gate-and-whats-next-after-stage-gate; and: gemba.dk/stage-gate/stage-gate-resources/
4. Source: Cooper, R.G.; see www.bobcooper.ca
5. Cooper, R. G., “Agile-Stage-Gate Hybrids: The next stage for product development” Research-Technology Management, Jan 2016, 59, 1, pp 1-9. http://www.bobcooper.ca/articles/agile-stage-gate
6. Cooper, R.G. & Sommer, A.F., “Agile-Stage-Gate: New idea-to-launch method for manufactured new products is faster, more responsive” Industrial Marketing Management, forthcoming 2016.
7. ‘See Beck et al., 2001: www.agilemanifesto.org
8. Cooper, R. G., “Agile-Stage-Gate Hybrids: The next stage for product development” Research-Technology Management, Jan 2016, 59, 1, pp 1-9. http://www.bobcooper.ca/articles/agile-stage-gate
12. Schwaber, K. & Sutherland, J.: The Scrum Guide, 2013, see: www.scrumguides.org/docs/scrumguide/v1/scrum-guide-us.pdf and Schwaber, K. Agile Project Management With Scrum, Microsoft, 2003.
13. Takeuchi H. & Nonaka, I. “The new new product development game”, Harvard Business Review, 1986, 64, 1, pp 137–146. https://hbr.org/1986/01/the-new-new-product-development-game and: The Knowledge-creating Company: How Japanese Companies Create the Dynamics of Innovation, 1995, Oxford University Press.
14. Vedsmand,T.: ‘Making Ideation a Part of the ‘Innovation Project Machine’, 2013, InnovationManagement.se, www.innovationmanagement.se/2013/02/04/making-ideation-a-part-of-the-innovation-project-machine/.
15. Cooper, R “What’s next After Stage-Gate”, Research Technology Management, 2014: www.bobcooper.ca/images/files/articles/0/3-Whats-Next-After-Stage-Gate.pdf and: Sommer et al., 2015: www.projectmanagement.com/content/attachments/jwoolcott_260515093635.pdf and Cooper, R. & Sommer, A., ”The Agile–Stage-Gate hybrid model: A promising new approach”, Journal of Product Innovation Management, 2016, on-line.
16. Cooper, R., webinar presentation May, 2016; also: Cooper, R.G. & Sommer, A.F., “Agile-Stage-Gate: New idea-to-launch method for manufactured new products is faster, more responsive,” Industrial Marketing Management, forthcoming 2016.
17. See: Sommer et al., 2015; and Cooper Sommer, 2016, in footnote 14.
18. Christensen, C.M. & Rayner, M.E., “The innovators solution”, Harvard Business School, 2003.
19. Buur, J. & Matthews, B., “Participatory innovation”, International Journal of Innovation Management, Sept. 2008, 12, 3, pp. 255–273. Participatory Innovation involves collecting experiences based on Lead-User Innovation, Participatory Design and Design Anthropology methods.
20. Ostervalder, A. & Pigneur, Y., Business model generation: A handbook for visionaries, game changers, and challengers, 2010, NY: John Wiley & Sons.
21. See Cooper & Sommer, endnote 15, 2016.