Prefer to listen to Chris Tacy as he tells you this story? Listen to the audio book of this article, here.

One of the strongly communicated beliefs of many so-called “Innovation” practitioners is that their approach to “Innovation” is superior because they use the Scientific Method. 

My perspective is that this is a common characteristic and belief that was inherited from Design Thinking. The problem is that many such practitioners at best hand wave at the Scientific Method, and in fact are often completely unaware of what it really is. They often have no understanding of how science actually works - thus completely eliminating the benefits of their practice. 

And this, again, is something I have observed being inherited from Design Thinking. The sins of the fathers and so on…

So… let’s start with what this “Scientific Method” thing really is.

“The scientific method is an empirical method for acquiring knowledge that has characterized the development of science since at least the 17th century. It involves careful observation, applying rigorous skepticism about what is observed, given that cognitive assumptions can distort how one interprets the observation. It involves formulating hypotheses, via induction, based on such observations; the testability of hypotheses, experimental and the measurement-based statistical testing of deductions drawn from the hypotheses; and refinement (or elimination) of the hypotheses based on the experimental findings.” Wikipedia

In other words, the Scientific Method is a repeatable model for learning the truth through a rigorous sequence of repeatable actions that depend on objective data for credibility.

This all sounds great.

And it sounds like it’s a perfect way to make “Innovation” efforts more pragmatic and practical and to as a result increase the odds of success.

But there is a problem in practice.

And the problem is in “Observation / question.”

So here is my observation… 

Most “Innovation” consultants and practitioners don’t start with Observations but instead with Opinions and Ideas. And this simply doesn’t work. And it’s really not the scientific method at all - it's just a predetermined set of perceptions.

What I see is people who start with their own idea, then back into the research topic area, then reverse engineer a hypothesis to support that idea or opinion. And then folks start running experiments and tests - not based on an Observation but rather on their own Idea or Opinion. 

Now… of course and predictably, most of the time this results in poor conclusions. At which point one of two things happens (both of which are, again, violations of the principles of the Scientific Method). When most practitioners get these poor outcomes from the conclusion phase the answer tends to be, “the problem was the testing” or excuses are made like “we need more time to test” or they just say “let’s try a different idea.”

So is it a shock that over 90% of all funded “Innovation” efforts fail?

The whole premise of the Scientific Method is that someone using it will “learn the truth.” Not that doing so will validate one’s own brilliance. Not that this will mean funding for a pet experiment. Saying “Scientific Method” doesn’t by itself grant the right to run whatever experiments are desired.

And the whole problem here seems to be that most people doing this kind of work just don’t seem to want the truth. 

At the beginning of all of this I said that this flaw… this cognitive mis-firing… this misuse of the Scientific Method was inherited from Design Thinking (along with many methods, dogmas, etc). Here is a good quote from a piece on Design Thinking and the Scientific Method that pretty clearly demonstrates the problem.

In a nutshell, the scientific method emphasizes experimentation, discovery, and inductive reasoning. You start by making observations, often through the use of experiments, and combine the results of those experiments with existing facts. - Conoor Behal

You’ll note that (in their telling of it), the Scientific Method may begin with Observations. But these Observations are the result of Experimentation. You in fact begin with Experimentation as a way to make Observations - that the Observations are the results of those Experiments. And existing facts are neither the bedrock all this is built on nor the lines we are coloring within… but rather merely supporting data (often cherry picked in practice).

So how is this wrong in practice? How does it play out in “Innovation”?

Let me tell you a (purely fictional) story 

A struggling large company comes to an “Innovation” practitioner and says, “please help us! Our sales of buggy whips are in decline and these new emerging technologies  threaten to put us out of business!!!” 

The practitioner (a smart and on-top of the trends thinker) says, “Oh! They really need to consider a Generative AI based marketplace for buggy whips and buggy whip accessories.” They then design a series of experiments.

The first challenge comes when they try to find subjects to test on. It turns out that there aren’t that many buggy whip buyers out there. But they soldier on and in the end meet their goal of “more than a dozen” subjects experimented with.

The data from these experiments show that the vast majority of buggy whip buyers have no interest in a buggy whip and buggy whip accessory marketplace. In fact, it looks rather like none plan to buy another buggy whip in their lifetime. But they are, in fact, intrigued by Generative AI.

So the practitioner designs another set of experiments, this time around Generative AI for buggy whip owners.

This time around it’s even hard to find test subjects. It almost seems like the customers are vanishing. But they continue to soldier on and test more than a dozen customers on Generative AI for buggy whip owners.

The data, sadly, is not promising. It turns out that everyone is excited about Generative AI - but none identify as buggy whip owners.

The practitioner, now desperate, says that the problem is the size of the audience being tested against and designs a new experiment - this time broadening their subjects outside of buggy whip owners and focusing on excitement about Generative AI and Transportation. And finally they hit gold! Everyone is very excited about how Generative AI is going to help with their car navigation and entertainment systems, Some are intrigued by the intersection of Generative AI and self-driving cars.

The practitioner now goes back to the client and says, “we have the answer! You need to pivot from buggy whips to Generative AI for cars. And we have the insights to back it up.”

That, my friends, is how you can abuse and misuse the Scientific Method to get to 90% fail rates in your “Innovation” efforts.

So... how do we fix this? How do you actually and honestly apply the Scientific Method to these so-called “Innovation” efforts, and if so, how?

I’ll start with the simplest (and yet the hardest and most important) step. We need to set our own egos aside and start by observing without preconceptions or desires or agendas. I know folks have great ideas. I know everyone doing this is super smart. But stop. Get outside. Talk to people. Observe others. We all need to put our own ideas aside (no matter how brilliant they may be). 

At Mach49 we do this by starting with Customer Development. It’s a great model and set of frameworks and tools for listening to customers and understanding their pain. But there are a number of other ways to get out of your own bubble and your team’s bubble and observe. 

Secondly, we will need to (again) put our egos aside, and when we measure results and report conclusions we must be honest with ourselves and ask the true hard questions. We have to set aside hopes and beliefs and desires and let the data and observations lead us to the truth. We cannot, as David Ogilvy famously said, “use research as a drunkard uses a lamppost – for support, but not for illumination.”

And if you are able to do this - if you are able to take your ego and your own needs and your own hopes out of the process and treat research as illumination and put in the hard work – then you can use these methods to find the truth.

And in “Innovation” as in all other things in life, the truth truly shall set you free.