The world seems to be awash with pithy advice on how to make good decisions:
- “Unsuccessful people make decisions based on their current situation; successful people make decisions based on where they want to be.”
- “Don’t base your decisions on the advice of those who don’t have to deal with the results.”
- “Good decisions come from experience, and experience comes from bad decisions.”
- “Unnecessary fear of a bad decision is a major stumbling block to good decisions.”
- “Indecision is a decision.”
One might think that with so much data available on nearly every conceivable issue, decision-making today would be easier than ever. But then there’s that pesky thing called information overload.
An excellent guide in navigating this challenge can be found in Decisions Over Decimals: Striking the Balance Between Intuition and Information by Christopher Frank, Paul Magnone, and Oded Netzer.
The authors have credentials that seem tailor-made for such a book. Frank is vice president of Global Marketplace Insights at American Express where he leads the communications and brand research, analytics group. He’s also an adjunct professor at Columbia University. Magnone is head of the Global Strategic Alliances group at Google. As a systems thinker and business builder, he previously worked at Deloitte and IBM. Netzer is vice dean of research and a professor at Columbia Business School. He's a world-renowned expert in data-driven decision-making and extracting meaningful insights from data.
So, what do these smart guys have to say about making smart decisions? Listen in on our conversation.
Rodger Dean Duncan: Quantitative Intuition (QI)TM, you say, can produce more effective and more efficient decisions. In a nutshell, how does it work?
Christopher Frank: Simply put, Quantitative Intuition, or QI, is the ability to make decisions with incomplete information via a three-prong approach. First, ask powerful questions. Second, put the data into context. Finally, synthesize (as opposed to summarizing) the information by combining the information with judgment.
With the abundance of data, there is the erroneous belief that we can achieve the perfect decision. However, the perfect decision does not exist. We still need to use intuition and judgment in decision-making. But it’s a different type of intuition—one that combines information with human judgment, which we call QI.
Oded Netzer: For many years at Columbia University we’ve been teaching classes on QI to executives. As part of the programs, we’ve asked executives to identify the aspect of decision-making they think represents the biggest gap in their organizations when it comes to making smarter data-driven decisions. Across thousands of executives, we’ve found that the biggest gap is not in having more data or a better analysis tool to crunch the numbers. The gaps lie in defining the essential question, generating meaningful insights, and converting these insights into action. The problem in today’s data-rich environment is not information, but rather the judgment to use it.
Duncan: What mindset adjustments are required of someone who wants to employ QI?
Paul Magnone: A decision represents change, and humans are not wired for change. Most of us retreat to comfort zones—some to data and others to gut instinct.
Great decision-makers judiciously explore opportunities with probing curiosity. They’re open to alternatives while being focused on essential outcomes. You also must get past the belief that you need to be a math expert to make sound, fact-based decisions. People avoid using quantitative analysis because they believe they won’t have the ability to navigate the data. The data is the means and not the end.
The QI decision-maker uses the combination of precision questioning, contextual analysis, and synthesis to see the whole situation to move forward despite incomplete information.
Duncan: What kind of biases affect people’s decision-making—for good or for ill?
Netzer: The beauty of the human race is that individuals are not robots. We all have opinions shaped by factors we’re unaware of, preconceptions we don’t know we have, and beliefs we don’t know we hold. The decisions we make will always be affected by bias. There are biases related to relying primarily on intuition and not using data, such as overconfidence and availability bias.
Overconfidence prevents leaders from turning to data to question their intuition, and availability bias directs decision-makers to the most easily accessible data. On the other hand, confirmation bias occurs when people look primarily for information that confirms their intuitive view.
To be able to foster and develop our quantitative intuition, we must be aware of our biases and the impact they have on us. Contrasting your intuition with data and paying careful attention to the cases in which the two disagree is a key to identifying potential biases. Creating diverse teams can help mitigate the effect of biases because while the data is factual, different people will see the same data from different perspectives.
Duncan: What role does skill in asking questions play in a person’s ability to make sound decisions?
Netzer: The smartest person in the room is not the one with an answer, but the person asking the powerful questions. If you seek bolder decisions, ask better questions.
Questioning is analogous to pulling threads on a sweater. Some loose threads will just come out; others can unravel the whole sweater. Questioning enables you to quickly pull threads to see which are superfluous, integral, or consequential. Questioning helps your team make inferences and connections about data and opens up viewpoints or analyses that are not apparent. This exploration mindset encourages trial and iteration. As a leader, you should strive to create a learning environment that fosters the question-learning loop. Building a team of questioners is the path to winning, to achieving the robust growth many companies seek.
Duncan: You write about a sequence of statements you refer to as IWIK—“I wish I knew.” You say IWIKs enable people to focus quickly on the most essential questions so they can prioritize their efforts to make efficient and effective decisions. Please give us an example of how this approach works.
Frank: Agile decision-making is grounded in how you think, not how hard you work. That starts with a deceptively simple, yet extremely powerful, question: “What do I wish I knew to make the best decision?” This question generates a sequence of statements we refer to as IWIK™, “I wish I knew.”
The key word here is “wish,” which grants permission for open exploration and not simply rehashing what is already known.
There are four parts to the IWIK process—Ask, Brainstorm, Capture, and Deliberate. This leads to uncovering the underlying questions that can expose the root cause at the epicenter of the puzzle that’s being solved. The output is a series of statements, all beginning with the reply, “I wish I knew …” The IWIK statements your colleagues or clients provide reveal a deep understanding of their actual needs. IWIKs enable you to focus on the essential question and prioritize your teams to make efficient and effective decisions. IWIK statements act as a catalyst to bring clarity to an issue.
Duncan: Systems thinking pioneer Russell L. Ackoff said, “We fail more often because we solve the wrong problem than because we get the wrong solution to the right problem.” What causes people to fall into that trap?
Magnone: This sums up what we have experienced on the front line of business. Too often, leaders expect the data to provide both the question and the answer. It’s your responsibility to uncover the essential question and then combine data with intuition to identify the answers.
A useful analogy to keep in mind is Lewis Carroll’s Alice in Wonderland, which involves Alice falling down a rabbit hole. At one point, Alice finds herself at a fork in the road, much like today’s decision-makers. Utterly confused, she asks the Cheshire Cat which path she should choose. “That depends a good deal on where you want to get to,” her feline consultant responds. “I don’t much care where so long as I get somewhere ...,” says Alice, at which point the cat interjects, “Then it doesn’t matter which way you go.” Think of Alice as an analyst, consultant, or data scientist walking in the thick forest of data, trying to find some useful insights but unsure of where she hopes to arrive or what that “somewhere” might even look like.
Very often, decision-makers are uncertain or vague about what they are looking for. It is common for business leaders to assert that they’ve got loads of data and to believe, therefore, that there must be some gems in it. And indeed, just like Alice, the analyst who walks long enough in the forest of data will inevitably find some interesting correlations or patterns. But without an end goal in mind, how much of that analysis will end up being truly relevant to what you’re trying to achieve? Unless you intentionally guide the process, most of these exploratory journeys lead nowhere. Pausing to think carefully about a problem and possible solutions is hard. It requires understanding the situation, having relevant acumen, and considering alternatives while avoiding a random walk.
Duncan: How can decision-makers become more skilled at “fiercely interrogating” the data they’re using to inform their decisions?
Netzer: At the heart of interrogation is the skill of asking the right questions, either through repetition or observation. We have developed habits that undermine our ability to solve a problem. We jump into solution mode, often confusing activity with impact. In our rush, we neglect to understand the data.
People consider data without taking the time to ask for context. To put data in context, you must always triangulate it by looking at it in (1) absolute terms, (2) over time, and (3) relative to what’s going on elsewhere. Data without context is dangerous; it leads to wrong conclusions and poor decisions.
But the main skill needed to interrogate data is not a technical one, it involves putting the data in the context of the business and asking yourself what surprised you about the data or analysis. By definition, surprises are a mix of intuition and information. When information does not match the intuition or the context, we get a surprise. It’s often exactly at these surprise points that the magic of meaningful insights occurs.
Duncan: What can leaders do to help cultivate a Quantitative Intuition culture in their organizations?
Frank: Decision-making is a team sport. Build a team composed of four roles—data scientists, data engineers, data translators, and data leaders. The current gap in the workforce is less about people with deep analytical skills and more about leaders who can lead them to make better decisions with analytics and judgment.
As we hire for QI skills, we should focus on leaders’ ability to ask precise questions, put the data in context by interrogating it, and synthesizing the information. These steps require asking powerful questions. Leaders should develop inquisitive teams that constantly ask questions as opposed to jumping directly into solution mode. Invest time and energy in visualization, with a focus on data translators who sit between the data and the business context.
Duncan: What question do you wish I had asked, but didn’t … and how would you respond?
Frank: I love this question since it mimics exactly the IWIK approach. An important question is “How do you recruit talent to lead in a data-driven world?” The typical structure of a job interview often involves the interviewer asking a stream of questions, and the candidate is expected to masterfully provide enlightening answers. This one-way process is inherently flawed. It fails to assess the most important growth skills—demonstrating an interrogative approach to problem-solving and a curious mindset.
These skills are particularly necessary for positions that involve making data-driven decisions, where the ability to be a fierce interrogator is essential. To assess a person’s interrogative mindset, we assert flipping the typical interview process on its head, opening the door for the candidate to be the interrogator and assessing their ability to ask powerful questions. An effective hiring process focuses on assessing a new set of skills around being a powerful questioner. In interviews, focus on people’s ability to move from the “what?” to the “so what?” and “now what?”