Measuring Innovation part 1: Frequently Used Indicators

One of the most common questions people ask me is how I measure innovation when conducting my research. The question echoes an underlying concern about how innovation can be captured and adequately measured. In this article I delineate the most frequently used innovation indicators, their strengths, and their drawbacks.

This article is the first part of a two part series on how to measure innovation in organizations. But let’s make it clear from the beginning. It is not easy to measure innovation since virtually all indicators risk missing some aspects of the process. For instance, patents or patent applications indicate some kind of technological progress but patents are seldom transformed into tangible products. In fact it looks bad for patents. 90 to 95 percent of all patents lack any market relevance and 99 percent fail to bring any profit to the firm (Stevens & Burley, 1997). Still, patenting is often necessary for many firms to keep their competitive advantage so clearly patents indicate something about the innovativeness of a firm. What about financial figures such as revenue? These risk being too broad since many other influences affect revenue besides innovation, including luck.

Below I summarize the most frequently used measures of innovation including their main advantages and drawbacks. For better clarity I divide them into three categories. Product/technology measures, financial measures, and subjective measures.

Product/technology measures

These indicators measure aspects of the innovation process which have the distal goal of transforming new ideas and technology into tangible products (goods or services).

Measure

Example study

Main advantage

Main drawback

New products or product improvements

Elenkov & Manev (2009)

Measures actual implementation

All products are not guaranteed to succeed

Patents or patent applications

Jung et al. (2008)

Measures technological progress

Patents are seldom realized

Patent citations

Makri & Scandura (2010)

Measures importance of patents

Patents may be self-cited

Invention disclosures or suggestions

Axtell et al. (2000)

Measures rate of idea generation

Ideas are seldom realized

Process innovations

West et al. (2003)

Measures improvements in processes and methods

Too much focus on processes is the ‘innovators dilemma’

Financial/market measures

These indicators measure aspects of the financial performance of an organization. Mainly in relationship to R&D spending and sales of new products (goods or services).

Measure

Example study

Main advantage

Main drawback

Ratio of sales of new products to total sales

Czarnitzki & Kraft (2004)

Indicator of success on market

Very broad, other factors confounds the measure

Ratio of sales of new products to R&D expenditures

Gumusluoglu & Ilsev (2009)

Indicator of R&D efficiency

Difficult to establish a valid baseline

Total R&D spending

García-Morales et al. (2008)

Easy to obtain

Does not indicate innovation efficiency

Number of employees in R&D

García-Morales et al. (2008)

Easy to obtain

Does not indicate innovation efficiency

New markets entered

Elenkov & Manev (2009)

Indicator of radical innovation

Roughly 60 percent of new products succeed

Subjective measures

Although the indicators above are among the most common when measuring innovation in firms, many of the activities that can be characterized as innovative risk being overlooked if innovation is measured solely using the broad searchlight of these quantitative measures. Activities like these can be labeled ‘dark innovation’ (Martin, 2012).  Examples include activities that are incremental (e.g., improvements in quality), involve little formal R&D (e.g. the concept of ‘skunk work’), and is seldom patented. One method to capture ‘dark innovation’ is to use subjective assessments.

Measure

Example study

Main advantage

Main drawback

Innovative work behavior (IWB)

De Jong & Den Hartog (2010)

Flexible, can measure any innovation activity

IWB’s does not unequivocally lead to tangible outcomes

Team innovativeness

Hurley & Hult (1998)

Flexible, can measure any innovation activity

Low correlations with number of implemented innovations

Organizational innovation

Chen et al. (2006)

Holistic assessment of the organization

Difficult to establish a valid baseline

Selection of measure is highly specific to the organization and depends on a number of factors. The relevance of indicators varies with technological domain and also depends on the specific product offering of the firm. Moreover, R&D functions of organizations may differ in degree of formalization which affects the degree to which ‘dark innovation’ takes place.

In my research which primarily concerns the innovation of individuals and teams in industrial (mainly automotive) R&D environments we have chosen a combination of quantitative indicators of innovation and subjective assessments of individuals’ innovative behaviors. This method covers outcomes of innovation as well as ‘dark innovation’ activities. In my next article we will look more closely at the elusive nature of ‘dark innovation’, as well as subjective assessments of innovation including what they measure and how they correlate with tangible outcomes.

By Leif Denti

About the author

Leif Denti is pursuing his doctoral degree of Psychology at the University of Gothenburg, Department of Psychology. His main research venue is how project leaders stimulate creativity and innovation in their project teams (project name: Management for Sweden). Leif Denti also works as a consultant at Prospero Technology Management. Leif Denti holds a licentiate degree in Psychology at the University of Gothenburg.

 

References

Axtell, C. M., Holman, D. J., Unsworth, K. L., Wall, T. D., & Waterson, P. E. (2000). Shopfloor innovation: Facilitating the suggestion and implementation of ideas. Journal of Occupational and Organizational Psychology, 73, 265-285.

Chen, G., Tjosvold, D., & Liu, C. (2006). Cooperative goals, leader people and productivity values: Their contribution to top management teams in China. Journal of Management Studies, 43, 1177-1200.

Czarnitzki, D., & Kraft, K. (2004). Firm leadership and innovative performance: Evidence from seven EU countries. Small Business Economics, 22, 325–332.

De Jong, J., & den Hartog, D. (2010). Measuring innovative work behavior. Creativity and Innovation Management, 19, 23-36.

Elenkov, D. S., & Manev, I. M. (2009). Senior expatriates leadership’s effects on innovation and the role of cultural intelligence. Journal of World Business, 44, 357–369.

García-Morales, V. J., Mathías-Reche, F., & Hurtado-Torres, N. (2008). Influence of transformational leadership on organizational innovation and performance depending on the level of organizational learning in the pharmaceutical sector. Journal of Organizational Change Management, 21, 188–212.

Gumusluoglu, L., & Ilsev, A. (2009). Transformational leadership and organizational innovation: The roles of internal and external support for innovation. Journal of Product Innovation Management, 26, 264–277.

Hurley, R. F., & Hult, T. M. (1998). Innovation, market orientation, and organizational learning: An integration and empirical examination. Journal of Marketing, 62, 42-54.

Jung, D., Wu, A., & Chow, C. W. (2008). Towards understanding the direct and indirect effects of CEO’s transformational leadership on firm innovation. The Leadership Quarterly, 19, 582–594.

Makri, M., & Scandura, T. A. (2010). Exploring the effects of creative CEO leadership on innovation in high-technology firms. The Leadership Quarterly, 21, 75–88.

Martin, B. R. (2012). Innovation studies: Challenging the boundaries. Sussex: Science and Technology Policy Research, University of Sussex

Scott, S. G., & Bruce, R. A. (1994). Determinants of innovative behavior: A path model of individual innovation in the workplace. Academy of Management, 37, 580–607.

Stevens, G. A., & Burley, J. (1997). 3,000 raw ideas = 1 commercial success! Research-Technology Management, May-June, 16-27.

West, M. A., Borril, C. S., Dawson, J. F., Brodbeck, F., Shapiro, D. A., & Haward, B. (2003). Leadership clarity and team innovation in health care. The Leadership Quarterly, 14, 393–410.

 

 

Ad

FOLLOW US

Find out how innovation can be learned.
Ad

RESOURCE DIRECTORY

IM services, training, tools & more.
Need an account? Sign up >
Ad