Framing the Critical Question: Insights from Survey Research

Campaign teams cover a lot of ground as they work with the sponsor of a collaborative innovation challenge. In this article, innovation architect Doug Collins makes the case that campaign teams should focus their energies on helping the sponsor develop the critical question that serves as the basis for convening the community. Forming the powerful question—the question that accurately reflects the sponsor’s intent and that resonates with the community—yields the greatest return on time spent in developing the campaign, relative to its ultimate success.

Get the question right

Developing a challenge for a collaborative innovation campaign can take months. The campaign team spends time during the period seeking commitment from people to serve in roles such as sponsor, moderator, and coach. They spend time developing the campaign mechanics, including when and how to engage the community. They cover a lot of ground with a lot of people.

In my experience the most important job that the campaign team performs during these months—the activity that most affects the ultimate success of the campaign—is helping the sponsor and their core group of advisors formulate the critical question upon which they want to invite the community to discuss and contribute their insights.

Get this part right and they achieve results, even if they permit other elements of the campaign to slip.

Get this part right and they achieve results, even if they permit other elements of the campaign to slip. Get this part wrong and they find themselves with an overabundance of invalid insights and ideas, the surplus driven by their flawless execution of the plan to engage the community.

Lamentably, too, I find that campaign teams do not spend enough time perfecting this critical element of the campaign. They hesitate to request the needed level of commitment, or engagement, from the sponsor. They lack the guidelines to help them develop the question properly. Yet, the essence of the practice of collaborative innovation lies in the discipline of framing and reframing the critical question, then engaging the community as they respond.

In this article I address the latter challenge: question formation. The campaign team, once they have a vision for what makes for a quality question, feels more confident in seeking the sponsor’s time to develop it. To this end, the article may help the campaign team make headway in resolving the former challenge, too.

Drawing lessons from the discipline of survey research

A number of valuable resources on the subject of challenge question formation exist, including most notably Getting to Innovation by Arthur Van Gundy. For this article, I draw primarily from Jack Fowler and his book, Designing Survey Questions. Fowler approaches the subject of question design from the more mechanical, empirical perspective of designing effective survey instruments for field research. The field of survey research has for years grappled with the challenge of proper question formation, recognizing the important role that question design plays in yielding valid results. Fowler, specifically, lays out a set of principles to guide one’s thinking on what makes for proper question design. I will relate them as follows to the design of challenge questions, as his guidance and insight maps well to the practice of collaborative innovation.

Principle: ask people about their firsthand experience

In survey research, asking people to speculate yields unreliable responses. People know if they have been a victim of crime. They have no reliable perspective on crime county-wide, for example. Relative to the practice of collaborative innovation, I always recommend grounding the enquiry to the community members in the reality of their own experience. Specifically, regardless of whether the challenge question is aspirational or practical in nature, we solicit their observations: What caused you, the contributor to have this idea?

If the nature of the challenge question is around improving customer experience, then we encourage them to reflect first on their experiences from having served customers. The core idea may prove to be highly speculative, which is welcome. However, the idea becomes compelling in this context when it ties to real-world experience (e.g., “I notice that customers must wait 30 minutes before they can receive a response. We can cut the wait time in half by doing X.”).

Principle: ask one question at a time

Van Gundy likewise offers the same “keep it simple” advice, with the additional guidance not to embed potential solutions in the enquiry. This principle falls under the umbrella of common sense. Yet, the campaign team, fearing the undesirable response, will often go to extreme lengths to preclude surprises, leading to constructions such as, “How might we reduce inventory carrying costs by 10%… in a way that involves hiring left-handed tuba players who spent their formative years in Hamburg?”

The remedy for this scenario is for the campaign team to ask themselves if they are, in fact, leading the witness—and then be merciless in cutting the lead. Likewise, be merciless in breaking apart compound questions.

Principle: word questions such that each respondent is answering the same enquiry

Clarity differs from comprehension. The following challenge question is straightforward: “How might we enter the education market?” However, community members are left to wonder whether the sponsor has the local preschool, vocational program, or university campus in mind.

Provide context ahead of the challenge question—not in terms of the optimal solution but rather the optimal target.

To this end, campaign teams are wise to provide context ahead of the challenge question—not in terms of the optimal solution but rather the optimal target (e.g., “please focus your enquiries on vocational programs, in particular”). The context can come in the form of a letter from the sponsor which serves double duty as an instrument of greater comprehension and reassurance that the organization condones their spending time to contribute to the campaign. Likewise, testing your question with a handful of colleagues who have not participated in its development helps you see likely points of ambiguity.

Organizations that have achieved a level of maturity in developing campaigns often invite the community to form the critical question together through approaches such as The World Café or Open Space. In doing so, the community provides its own context and meaning.

Principle: clearly communicate expectations

Clearly communicate to all community members the kind of response that constitutes an adequate answer to a question. Adhering to this principle confounds both market researchers and campaign teams. The team devotes the necessary time to develop a clear, contextualized enquiry to the community. A member, pressed for time, short hands their response, providing only the germ of what could be a compelling idea, leaving the campaign team wondering what to make of the contribution. Here, I strongly recommend using the observation, implication, application approach developed by consultancy Innovation Focus and described in a previous article.

In brief, seek their observations. What did they experience that caused them to have this idea? Ask them to draw implications. What does the idea mean to the organization? To the intended recipients or beneficiaries? Then, ask them for their perspective on next steps. What should the organization do next—tomorrow—to explore and realize the benefit from the idea?

Campaign teams hesitate to burden the innovator for fear of losing their contribution, altogether. My experience suggests otherwise. What I find is that people, inspired to have and to share an idea, will willingly and joyfully make the effort to fully convey their insight. It’s a commitment they make as an expression of the personal mastery they exhibit with respect to the topic at hand. Campaign teams have the opportunity to engage in powerful ways with the innovator when they respect their commitment. Doing so puts them in the best position to fully explore the potential that the idea represents.

In closing, people who practice collaborative innovation can enhance their own mastery by reflecting on the insights and experiences gained from related, enquiry-led disciplines such as survey research. These practitioners, too, struggle with framing the critical question, providing relevant context, and deriving essential meaning from the responses they elicit. They have in turn developed empirically sound guidelines for improving question design.

By Doug Collins

About the Author:

Doug Collins serves as an innovation architect. He has served in a variety of roles in helping organizations navigate the fuzzy front end of innovation by creating forums, venues, and approaches where the group can convene to explore the critical question. He today works at Spigit, Inc., where he consults with Fortune 1000 clients on realizing their vision for achieving leadership in innovation by applying social media and ideation markets in blended virtual and in-person communities.

Previously, Doug formed and led a variety of front end initiatives, including executive advisory programs for industry influencers, early adopter programs for lead users, corporate strategic planning, and structured explorations of new market and product opportunities. Before joining Spigit, Doug worked at Harris Corporation and at Structural Dynamics Research Corporation which is now part of Siemens Corporation.
  • Michael Moon | CEO | GISTICS

    I really, really like this Framework and argument.

    I would like to add additional support, emphasizing the underlying cognitive dimensions of questions, planning, and collaboration.

    1) Much of the newest insights about cognition derive from neural computational imaging of individuals performing various tasks, identifying what portions of the brain physically light up with increased neuron-firing. 

    Guess what? 95% of what goes on up there falls in the area (according George Lakoff, one of the leading pioneers in the area of computational neural linguistics), of the Cognitive Unconscious occurs below conscious awareness.

    2) So what goes on there in the Cognitive Unconscious? Plenty. However here are a couple highlights of what’s important to know: 

    66% cognitive processes include a visualization of some type. So… make your conversations as visuals possible, using a sketch board or shared whiteboard.

    We think and feel primarily by means of metaphors; language then interprets metaphors as rational thought; our deepest metaphors arise from life experiences and how we have framed those experiences (a whole library of books on frames!)

    Narratives, stories with the moral implication (where moral implies whatever causes well-being or the activation of dopamine circuits), represent the most powerful cognitive forces at play in decision-making and collaboration.

    To be an adult and enculturated means that we have a well-developed inventory of narratives. If fact cultural literacy means nothing more than having a rather complete set of narratives by which to instantly interpret a complex, dynamic world of interpersonal relations and transactions.

    People love strong, well-framed, well-told narratives BECAUSE it is how we think at the most basic levels of cognition.

    A great story will trump hard data in most social situations; most elections demonstrate this fact.

    So always, always start a project with a great story, told well, about why you’re doing or considering the project.

    Start your narrative with an inspired, open-ended question!

    3) When asked a question, the human mind cannot resist answering it to the best of its ability. When asked question, our mind instantly fills in the best answer (where best reflects many things such as who’s asking and what can mine for as their motive for asking, etc.) ALWAYS!!!

    A concise essential question can not only bring clarity to everyone and here it, if embedded within a strong narrative and delivered with moral authority, one essential question can transform the very circumstances of its asking and all those participating in those circumstances.

    With respect to innovating—which by formal definition entails adding distinctive capability to how a firm attracts, serves, or keeps profitable customers for life OR how it AND its partners enhance their customer’s experience of value, quality, or satisfaction in the  consideration, purchase, and disposal of a firms products or cessation of using a firm’s service—I have found few preliminary questions that really help move everything forward:

    What one thing do our customers (or stakeholders) love about us and what we bring to them?

    Why we doing this new project? Why are we evening considering this? What’s the purpose and guiding principles for deciding whether or not proceed?

    What it looked like you we produced were pulled off a complete, over-the-top success?

    Who would I and each of the stakeholders become with a complete success? How would I have to grow, or expand to include what new skills or mindset?

    Working backwards from a fully realized future success, what I call a  Narrative Futureproof, what comprise all of the essential complements, roles and accountabilities, sequences and handoffs, and probable derailments and their get-back-on-track procedures?

    This is just the start.

  • Doug Collins


    The concepts of visuals and narratives resonate with me.

    I think sometimes we equate — perhaps from our childhood — that saying, “let me tell you a story” with “let me tell you a tall tale.”

    In reality, what you’re saying is that the narrative — in our adulthood — can be the most effective way to get at essential truths.


  • Michael Moon | CEO | GISTICS

    Yes, exactly. As mature adults, we rely upon metaphors that we constructed as a 4-year old. Example: the sun rising in the east. Nice metaphor; only not correct. The Sun remains a stationery object around which the earth revolves; more accurately, the “earth rises” in the morning.

    More one implication for visuals and narratives: storyboarding  and visual storytelling.  I have used strong and often multi-dimensional visuals to convey the normally hard stuff of a complex innovation: context, hierarchy, flow, cause-effect, correspondence, and interrelatedness.

    In an ideal situation, we develop a graphic explanation of the to-be futurestate of an innovation that so resonates with the CEO (usually because he or she starts using it to communicate a complex dimension of his) that the graphic becomes an institutional icon for change and the CEO’s mandate.

    If you accept this premise (I have!), then you’d really want to co-create or at least co-edit your core graphic explanation of strategic innovation with the CEO or your executive sponsor!

  • Daniel

    If you haven’t read “Tell to Win” by Peter Guber, it is well worth the time.

    The book is about creating a purposeful narrative to create change or incite action.  Very interesting read with the scientific rational accompanied with the stories. 

    Effectively…  Powerpoints, Data, and having the right solution or question are not enough with out a purposeful narrative.

  • Rob3365


    Thank you for your post – incredibly insightful and useful.  

  • Pingback: Los 50 mejores artículos de Innovación Colaborativa V.1 2012 (En progreso) «