Researchers in recent years have exhaustively catalogued and chronicled the biases that affect our decisions. We all know the havoc that biased decisions can wreak. From misguided beliefs about the side effects of vaccinating our children, to failures in analysis by our intelligence community, biases in decision making contribute to problems in business, public policy, medicine, law, education, and private life.
Researchers have also long searched for ways to train people to reduce bias and improve their general decision making ability – with little success. Traditional training, designed to debias and improve decision-making, is effective in specific domains such as firefighting, chess, or weather forecasting. But even experts in such areas fail to apply what they’ve learned to new areas. Weather forecasters, for instance, are highly accurate when predicting the chance of rain, but they are just as likely as untrained novices to show bias when making other kinds of probability estimates, such as estimating how many of their answers to basic trivia questions are correct.
Because training designed to improve general decision making abilities has not previously been effective, most efforts to debias people have focused on two techniques. The first is changing the incentives that influence a decision. Taxing soda, for example, in the hopes that the increased cost will dissuade people from buying it. The second approach involves changing the way information for various choices is presented or choices are made, such as adding calorie information to fast-food menus or offering salad as the default side order to entrées instead of French fries. However, these methods are often not always effective, and when effective, only affect specific decisions, not decision-makers’ ability to make less biased decisions in other situations.
My research collaborators and I wondered if an interactive training exercise might effectively debias decision-makers. (The team included Boston University’s Haewon Yoon, City University London’s Irene Scopelliti, Leidos’ Carl W. Symborski, Creative Technologies, Inc.’s James H. Korris and Karim Kassam, a former assistant professor at Carnegie Mellon University.) So we spent the past four years developing two interactive, “serious” computer games to see if they might substantially reduce game players’ susceptibility to cognitive bias.
There was scant evidence that this kind of one-shot training intervention could be effective, and we thought our chances of success were slim. But, as we report in a paper just published in Policy Insights in the Behavioral and Brain Sciences, the interactive games not only reduced game players’ susceptibility to biases immediately, those reductions persisted for several weeks. Participants who played one of our games, each of which took about 60 minutes to complete, showed a large immediate reduction in their commission of the biases (by more than 31%), and showed a large reduction (by more than 23%) at least two months later.
The games target six well-known cognitive biases. Though these biases were chosen for their relevance to intelligence analysis, they affect all kinds of decisions made by professionals in business, policy, medicine, and education as well. They include:
Bias blind spot – seeing yourself as less susceptible to biases than other peopleConfirmation bias – collecting and evaluating evidence that confirms the theory you are testingFundamental attribution error – unduly attributing someone’s behavior to enduring aspects of that person’s disposition rather than to the circumstance in which the person was placedAnchoring – relying too heavily on the first piece of information considered when making a judgmentProjection – assuming that other people think the same way we doRepresentativeness – relying on some simple and often misleading rules when estimating the probability of uncertain eventsWe ran two experiments. In the first experiment, involving 243 adult participants, one group watched a 30-minute video, “Unbiasing Your Biases,” commissioned by the program sponsor, the Intelligence Advanced Research Projects Activity (IARPA), a U.S. research agency under the Director of National Intelligence. The video first defined heuristics – information-processing shortcuts that produce fast and efficient, though not necessarily accurate, decisions. The video then explained how heuristics can sometimes lead to incorrect inferences. Then, bias blind spot, confirmation bias, and fundamental attribution error were described and strategies to mitigate them were presented.
Another group played a computer game, “Missing: The Pursuit of Terry Hughes,” designed by our research team to elicit and mitigate the same three cognitive biases. Game players make decisions and judgments throughout the game as they search for Terry Hughes – their missing neighbor. At the end of each level of the game, participants received personalized feedback about how biased they were during game play. They were given a chance to practice and they were taught strategies to reduce their propensity to commit each of the biases.
We measured how much each participant committed the three biases before and after the game or the video. In the first experiment, both the game and the video were effective, but the game was more effective than the video. Playing the game reduced the three biases by about 46% immediately and 35% over the long term. Watching the video reduced the three biases by about 19% immediately and 20% over the long term.
In a second experiment, involving 238 adult participants, one group watched the video “Unbiasing Your Biases 2” to address anchoring, projection, and representativeness. Another group played the computer detective game “Missing: The Final Secret,” in which they were to exonerate their employer of a criminal charge and uncover criminal activity of her accusers. Along the way, players made decisions that tested their propensity to commit anchoring, projection, and representativeness. After each level of the game, their commission of those biases was measured and players were provided with personalized feedback, practice, and mitigation strategies.
Again, the game was more effective than the video. Playing the game reduced the three biases by about 32% immediately and 24% over the long term. Watching the video reduced the three biases by about 25% immediately and 19% over the long term.
The games, which were specifically designed to debias intelligence analysts, are being deployed in training academies in the U.S. intelligence services. But because this approach affects the decision maker rather than specific decisions, such games can be effective in many contexts and decisions – and with lasting effect. (A commercial version of the games is in production.)
Games are also attractive because once such approaches are developed, the marginal costs of debiasing many additional people are minimal. As this and other recent work suggests, such interactive training is a promising addition to the growing suite of techniques that improve judgment and reduce the costly mistakes that result from biased decision making.