Know Thyself: The Science of Self-Awareness - Stephen M Fleming 2021

Notes

Preface

1. Linnaeus (1735); Flavell (1979); Nelson and Narens (1990); Metcalfe and Shimamaura (1996).

2. Nestor (2014).

3. Shimamura (2000); Fleming and Frith (2014).

4. The MetaLab, https://metacoglab.org.

5. Comte (1988).

6. Descartes (1998).

7. Mill (1865).

8. Dennett (1996).

9. From a BBC interview with James Mossman, published in Vladimir Nabokov, Strong Opinions (New York: Vintage, 1990).

10. Hamilton, Cairns, and Cooper (1961).

11. Renz (2017).

12. Baggini (2019); Ivanhoe (2003).

13. Dennett (2018).

14. The terminology used by scientists and philosophers studying self-awareness and metacognition can get confusing at times. In this book, I use the terms metacognition and self-monitoring to refer to any process that monitors another cognitive process, such as realizing we have made an error in solving a math problem. Self-monitoring and metacognition may sometimes occur unconsciously. I reserve the term self-awareness for the ability to consciously reflect on ourselves, our behavior, and our mental lives. Some psychologists restrict the term self-awareness to mean bodily self-awareness, or awareness of the location and appearance of the body, but here I am generally concerned with awareness of mental states.

Chapter 1: How to Be Uncertain

1. Jonathan Steele, “Stanislav Petrov Obituary,” The Guardian, October 11, 2017, www.theguardian.com/world/2017/oct/11/stanislav-petrov-obituary.

2. Green and Swets (1966).

3. The seeds of Bayes’s rule were first identified by the eleventh-century Arabic mathematician Ibn al-Haytham, developed by English clergyman and mathematician Thomas Bayes in 1763, and applied to a range of scientific problems by the eighteenth-century French mathematician Pierre-Simon Laplace. See McGrayne (2012).

4. Felleman and Van Essen (1991); Zeki and Bartels (1998).

5. Clark (2013); Clark (2016); Craik (1963); Friston (2010); Helmholtz (1856); Gregory (1970); Hohwy (2013).

6. Kersten, Mamassian, and Yuille (2004); Ernst and Banks (2002); Pick, Warren, and Hay (1969); Bertelson (1999); McGurk and MacDonald (1976).

7. Born and Bradley (2005); Ma et al. (2006).

8. Apps and Tsakiris (2014); Blanke, Slater, and Serino (2015); Botvinick and Cohen (1998); Della Gatta et al. (2016); Seth (2013).

9. Kiani and Shadlen (2009); Carruthers (2008); Insabato, Pannunzi, and Deco (2016); Meyniel, Sigman, and Mainen (2015).

10. Smith et al. (1995).

11. It’s possible that alternative explanations that do not require tracking uncertainty could account for animals’ behavior in these experiments. For instance, when the third lever is introduced to Natua, there are now three responses: low tone, high tone, and “don’t know” (the opt-out response). After a while, Natua might learn that pressing the low or high keys when the tone is in the middle will often lead to a penalty for getting the wrong answer, and no fish to eat. The opt-out response is less risky, allowing him to move on swiftly to another trial where fish are on the table again. He might just be following a simple rule, something like, “When the middle tone occurs, press the opt-out lever,” without feeling uncertain about whether he’s likely to get the answer right. Carruthers (2008).

12. Kornell, Son, and Terrace (2007); Shields et al. (1997); Kepecs et al. (2008); Fujita et al. (2012). Six pigeons and two out of three bantam chickens were more likely to use the risky option when they were correct on a visual search task. Two pigeons also showed consistent generalization of this metacognitive ability to new sets of colors.

13. Goupil and Kouider (2016); Goupil, Romand-Monnier, and Kouider (2016).

14. The psychologist Josep Call offers the following summary: “I think that it is perhaps fair to say that the field has entered a sort of arms race in which increasingly elaborated non-metacognitive explanations are met with ever more sophisticated empirical evidence which in turn generate increasingly more complex non-metacognitive explanations.” Call (2012); Hampton (2001).

15. Beran et al. (2009).

16. Meyniel, Schlunegger, and Dehaene (2015).

17. The Hungarian mathematician Abraham Wald developed the theory of sequential analysis while working for the US government during World War II. Turing independently developed similar methods as part of the Banburismus process, which remained classified by the UK government until the 1980s. Hodges (1992); Wald (1945); Gold and Shadlen (2002).

18. Desender, Boldt, and Yeung (2018); Desender et al. (2019).

19. Bayesian inference is straightforward in restricted situations with only a few hypotheses. But when the problem becomes unconstrained, there is an explosion of possible dimensions along which we must estimate probabilities, making it rapidly intractable. A fast-moving research field spanning both AI and cognitive science is working on ever more ingenious approximations to Bayesian inference, and it is possible that similar approximations are used by the brain.

Chapter 2: Algorithms for Self-Monitoring

1. Allostasis refers to the process of predicting how homeostasis will need to be adjusted: “Stability through change.” Conant and Ashby (1970); Sterling (2012).

2. Clark (2016); Hohwy (2013); Pezzulo, Rigoli, and Friston (2015); Gershman and Daw (2012); Yon, Heyes, and Press (2020).

3. Badre and Nee (2018); Passingham and Lau (2019).

4. Michael Church, “Method & Madness: The Oddities of the Virtuosi,” The Independent, March 12, 2008, www.independent.co.uk/arts-entertainment/music/features/method-madness-the-oddities-of-the-virtuosi-794373.html.

5. Logan and Zbrodoff (1998); Logan and Crump (2011).

6. Crump and Logan (2010); Logan and Crump (2009).

7. Reynolds and Bronstein (2003).

8. Fourneret and Jeannerod (1998).

9. Diedrichsen et al. (2005); Schlerf et al. (2012). There is an alternative view in which this copy is not secondary to the main command—it (or at least part of it) is the command. This is known as active inference. At the heart of active inference is a deep symmetry between perceptual and motor prediction errors. Perceptual prediction errors alter our model of the world; in turn, motor or “proprioceptive” prediction errors cause our muscles to move to shape our limbs to be in line with our predictions. In other words, we induce an error by saying, “I want to (expect to) be over there,” and our motor system snaps into line. Clark (2013); Friston (2010); Adams, Shipp, and Friston (2013); Friston et al. (2010).

10. Blakemore, Wolpert, and Frith (2000); Shergill et al. (2003); Wolpert and Miall (1996).

11. Rabbitt (1966); Rabbitt and Rodgers (1977); Hasbroucq et al. (1999); Meckler et al. (2017).

12. Gehring et al. (1993); Dehaene, Posner, and Tucker (1994); Fu et al. (2019).

13. Goupil and Kouider (2016).

14. Schultz, Dayan, and Montague (1997). Associative learning comes in different forms. In “classical” or Pavlovian conditioning, anticipatory responses come to be associated with a stimulus or cue. In “operant” or instrumental conditioning, the animal or human needs to perform an action in order to get a reward.

15. Seymour et al. (2004); O’Doherty et al. (2003); Sutton and Barto (2018). Prediction errors are a key mathematical variable needed to train learning algorithms in a branch of computer science known as reinforcement learning (RL). RL suggests that when learning is complete, no additional dopamine needs to be released, just as Schultz found: the monkey has come to expect the juice after the light, and there is no more error in his prediction. But it also predicts that if the juice is unexpectedly taken away, the baseline dopamine response dips—a so-called negative prediction error. This was also borne out by the neuronal recordings.

16. Another way of thinking about the role of the dACC and signals like the ERN is that they signal intermediate progress on the way toward obtaining a more concrete or explicit reward. Botvinick, Niv, and Barto (2009); Shidara and Richmond (2002); Ribas-Fernandes et al. (2011).

17. Gadagkar et al. (2016); Hisey, Kearney, and Mooney (2018).

18. There is unlikely to be a sharp division between these different levels. For instance, the ERN is itself modulated by the smoothness of the action we make to get to the target. Torrecillos et al. (2014).

19. See Stephen M. Fleming, “False Functional Inference: What Does It Mean to Understand the Brain?” Elusive Self (blog), May 29, 2016, https://elusiveself.wordpress.com/2016/05/29/false-functional-inference-what-does-it-mean-to-understand-the-brain/; and Jonas and Kording (2017); Marr and Poggio (1976).

20. Another note on terminology is useful here. The philosopher Joëlle Proust distinguishes between procedural and analytic metacognition: procedural metacognition is based on lower-level feelings of fluency that may or may not be conscious, whereas analytic metacognition is based on reasoning about one’s own competences. Others, for instance Peter Carruthers, deny that implicit monitoring and control qualify as metacognition because they can be explained without appeal to meta-representation. Still others, such as Josef Perner, accept the primacy of meta-representation as a starting point for thinking about metacognition, but are willing to allow a gradation of mental processes that are intermediate in level between implicit monitoring and full-blown, conscious meta-representation. Perner (2012); Proust (2013); Carruthers (2008).

Chapter 3: Knowing Me, Knowing You

1. Aubert et al. (2014); McBrearty and Brooks (2000); Sterelny (2011).

2. Ryle (2012).

3. Carruthers (2009); Carruthers (2011); Fleming and Daw (2017); Thornton et al. (2019).

4. Baron-Cohen, Leslie, and Frith (1985); Wimmer and Perner (1983).

5. Hembacher and Ghetti (2014).

6. Bretherton and Beeghly (1982); Gopnik and Astington (1988); Flavell (1979); Rohwer, Kloo, and Perner (2012); Kloo, Rohwer, and Perner (2017); Filevich et al. (2020).

7. Lockl and Schneider (2007); Nicholson et al. (2019); Nicholson et al. (2020).

8. Darwin (1872); Lewis and Ramsay (2004); Kulke and Rakoczy (2017); Onishi and Baillargeon (2005); Scott and Baillargeon (2017); Paulus, Proust, and Sodian (2013); Wiesmann et al. (2020).

9. Courage, Edison, and Howe (2004). An alternative perspective is that the mirror test is probing a distinct (and possibly nonconscious) ability to use mirrors appropriately (as we do effortlessly when shaving or doing our hair) without requiring self-awareness. Heyes (1994); Chang et al. (2017); Kohda et al. (2019).

10. Bretherton and Beeghly (1982); Gopnik and Astington (1988).

11. Lewis and Ramsay (2004).

12. Call and Tomasello (2008); Kaminski, Call, and Tomasello (2008); Butterfill and Apperly (2013); Heyes (2015); Krupenye and Call (2019); Premack and Woodruff (1978).

13. Herculano-Houzel, Mota, and Lent (2006); Herculano-Houzel et al. (2007).

14. Birds also appear to have a primate-like scaling law. Dawkins and Wong (2016); Herculano-Houzel (2016); Olkowicz et al. (2016).

15. This raises an obvious question. We might have big heads, but we certainly don’t have the biggest brains around. What about huge but evolutionarily distant species, such as elephants or whales? Herculano-Houzel found that, in fact, an African elephant brain is not just larger than the human brain, it also has three times the number of neurons. At first glance, this would appear to be a spanner in the works of the theory that humans have unusually large numbers of neurons compared to other species. But it turns out that the vast majority—98 percent—of the elephant’s neurons are located in its cerebellum, not the cortex. As we saw in the previous chapter, it is likely that the cerebellum acts as a suite of autopilots, keeping action and thought on track, but (in humans at least) not generating any kind of self-awareness. It is possible that the elephant needs such a large cerebellum due to its complex body plan and trunk that requires a lot of fine motor management. The African elephant is the exception that proves the rule of human uniqueness; humans retain a cortical neuronal advantage relative to any species tested to date. Herculano-Houzel et al. (2014).

16. Ramnani and Owen (2004); Mansouri et al. (2017); Wallis (2011); Semendeferi et al. (2010).

17. Jenkins, Macrae, and Mitchell (2008); Mitchell, Macrae, and Banaji (2006); Ochsner et al. (2004); Kelley et al. (2002); Northoff et al. (2006); Lou, Changeux, and Rosenstand (2017); Summerfield, Hassabis, and Maguire (2009).

18. Lou et al. (2004).

19. Shimamura and Squire (1986).

20. Janowsky et al. (1989); Schnyer et al. (2004); Pannu and Kaszniak (2005); Fleming et al. (2014); Vilkki, Servo, and Surma-aho (1998); Vilkki, Surma-aho, and Servo (1999); Schmitz et al. (2006); Howard et al. (2010); Modirrousta and Fellows (2008).

21. Nelson et al. (1990).

22. Kao, Davis, and Gabrieli (2005).

23. Amodio and Frith (2006); Vaccaro and Fleming (2018).

24. Armin Lak and Adam Kepecs have shown that neuronal firing in the rat frontal cortex predicts how long they are willing to wait for a reward for getting the answer right on a decision-making task—a marker of implicit metacognition. Inactivating this same region by infusing a drug known as muscimol impairs their ability to wait, but not to make the initial decision. In this aspect, the rodents in Lak and Kepecs’s study are similar to humans with damage to the prefrontal cortex: their cognition was intact, but their metacognition was impaired. Other work in monkeys has shown that neurons in the parietal and frontal lobes and thalamus track uncertainty about the evidence in support or different decisions, such as whether a stimulus is moving to the left or right—just as Turing’s equations tracked evidence for or against a particular Enigma hypothesis. Lak et al. (2014); Middlebrooks and Sommer (2012); Kiani and Shadlen (2009); Miyamoto et al. (2017); Miyamoto et al. (2018); Komura et al. (2013).

25. Mesulam (1998); Krubitzer (2007).

26. Margulies et al. (2016); Baird et al. (2013); Christoff et al. (2009); Passingham, Bengtsson, and Lau (2010); Metcalfe and Son (2012); Tulving (1985).

27. Herculano-Houzel (2016).

Chapter 4: Billions of Self-Aware Brains

1. Freud (1997); Mandler (2011).

2. Like many potted histories, this one is oversimplified. It is clear that the first psychologists studying subjective aspects of the mind also took behavior seriously, and Wundt himself ended up being one of the harshest critics of research on introspection conducted by his students Titchener and Külpe. Conversely, research on animal behavior was well underway before the advent of behaviorism (Costall, 2006). There were also some early glimmers of modern approaches to quantifying the accuracy of self-awareness: in a classic paper that was ahead of its time, the Victorian psychologists Peirce and Jastrow proposed a mathematical model of metacognition, suggesting that m = clog where m denoted the confidence level of the subject, p is the probability of an answer being right, and c is a constant (Peirce and Jastrow, 1885). This equation stated that subjects’ confidence goes up in proportion to the logarithm of the probability of being right—an assertion supported by recent experiments. Van den Berg, Yoo, and Ma (2017).

3. Hart (1965).

4. There comes a point at which bias and sensitivity collide—if I am always 100 percent confident, then I will tend to have a high bias and low sensitivity. Clarke, Birdsall, and Tanner (1959); Galvin et al. (2003); Nelson (1984); Maniscalco and Lau (2012); Fleming and Lau (2014); Fleming (2017); Shekhar and Rahnev (2021).

5. Fleming et al. (2010).

6. Poldrack et al. (2017).

7. Yokoyama et al. (2010); McCurdy et al. (2013). See also Fleming et al. (2014); Hilgenstock, Weiss, and Witte (2014); Miyamoto et al. (2018); Baird et al. (2013); Baird et al. (2015); Barttfeld et al. (2013); Allen et al. (2017); Rounis et al. (2010); Shekhar and Rahnev (2018); Qiu et al. (2018).

8. Semendeferi et al. (2010); Neubert et al. (2014).

9. Cross (1977); Alicke et al. (1995). In a phenomenon known as the Dunning-Kruger effect, after its discoverers, overconfidence biases are most pronounced in those who perform poorly (Dunning, 2012; Kruger and Dunning, 1999). Kruger and Dunning propose that low performers suffer from a metacognitive error and not a bias in responding (Ehrlinger et al., 2008). However, it is still not clear whether the Dunning-Kruger effect is due to a difference in metacognitive sensitivity, bias, or a mixture of both. See Tal Yarkoni, “What the Dunning-Kruger Effect Is and Isn’t,” [citation needed] (blog), July 7, 2010, https://talyarkoni.org/blog/2010/07/07/what-the-dunning-kruger-effect-is-and-isnt; and Simons (2013).

10. Ais et al. (2016); Song et al. (2011).

11. Mirels, Greblo, and Dean (2002); Rouault, Seow, Gillan, and Fleming (2018); Hoven et al. (2019).

12. Fleming et al. (2014); Rouault, Seow, Gillan, and Fleming (2018); Woolgar, Parr, and Cusack (2010); Roca et al. (2011); Toplak, West, and Stanovich (2011); but see Lemaitre et al. (2018).

13. Fleming et al. (2015); Siedlecka, Paulewicz, and Wierzchoń (2016); Pereira et al. (2020); Gajdos et al. (2019).

14. Logan and Crump (2010).

15. Charles, King, and Dehaene (2014); Nieuwenhuis et al. (2001); Ullsperger et al. (2010).

16. Allen et al. (2016); Jönsson, Olsson, and Olsson (2005).

17. De Gardelle and Mamassian (2014); Faivre et al. (2018); Mazancieux et al. (2020); Morales, Lau, and Fleming (2018). There are also intriguing exceptions to domain-generality that deserve further study. First of all, just because metacognition appears to be domain-general from the vantage point of behavior does not mean that different metacognitive abilities depend on the same neural circuitry (see, for example, McCurdy et al., 2013; Baird et al., 2013; Baird et al., 2015; Fleming et al., 2014; Ye et al., 2018). Second, some modalities do appear to be metacognitively special—Brianna Beck, Valentina Peña-Vivas, Patrick Haggard, and I have found that variability in metacognition about painful stimuli does not predict metacognition for touch or vision, despite the latter two abilities being positively correlated. Beck et al. (2019).

18. Bengtsson, Dolan, and Passingham (2010); Bandura (1977); Stephan et al. (2016); Rouault, McWilliams, Allen, and Fleming (2018); Will et al. (2017); Rouault, Dayan, and Fleming (2019); Rouault and Fleming (2020).

19. Bang and Fleming (2018); Bang et al. (2020); Fleming and Dolan (2012); Fleming, Huijgen, and Dolan (2012); Gherman and Philiastides (2018); Passingham, Bengtsson, and Lau (2010); De Martino et al. (2013); Fleming, van der Putten, and Daw (2018).

20. Heyes and Frith (2014); Heyes (2018).

21. Pyers and Senghas (2009); Mayer and Träuble (2013).

22. Hughes et al. (2005). Similar studies of the genetics of metacognition are rare. David Cesarini at New York University studied 460 pairs of Swedish twins and asked them to carry out a twenty-minute test of general cognitive ability. Before taking the test, each twin also rated how they believed they would rank relative to others—a measure of over- or underconfidence. The study found that genetic differences explained between 16 percent and 34 percent of the variation in overconfidence (Cesarini et al., 2009). Similar results were obtained in a larger study of more than 7,500 children in the UK by Corina Greven, Robert Plomin, and colleagues at Kings College London. They collected both confidence data—How good do the children think they are at English, science, and mathematics?—and also measures of IQ and actual success at school. Strikingly, the results showed that around half the variance in children’s confidence level was influenced by genetic factors—around the same degree as the genetic influence on IQ itself (Greven et al., 2009). So far, genetic studies have only examined our overall sense of confidence in our abilities, and none have quantified metacognitive prowess using the tools described in this chapter. It would be fascinating to apply similar techniques to look at variation in metacognitive sensitivity: Does our genetic makeup also predict how well we know our own minds? Or is metacognition more like mindreading, a skill that is less tied to our genes and more influenced by the rich thinking tools we acquire from our parents and teachers?

23. Heyes et al. (2020).

24. Weil et al. (2013).

25. Blakemore (2018); Fandakova et al. (2017).

26. David et al. (2012).

27. Fotopoulou et al. (2009).

28. Marsh (2017).

29. Burgess et al. (1998); Schmitz et al. (2006); Sam Gilbert and Melanie George, “Frontal Lobe Paradox: Where People Have Brain Damage but Don’t Know It,” The Conversation, August 10, 2018, https://theconversation.com/frontal-lobe-paradox-where-people-have-brain-damage-but-dont-know-it-100923.

30. Cosentino (2014); Cosentino et al. (2007); Moulin, Perfect, and Jones (2000); Vannini et al. (2019); Agnew and Morris (1998); Morris and Mograbi (2013).

31. Johnson and Raye (1981); Simons, Garrison, and Johnson (2017).

32. Frith (1992); Knoblich, Stottmeister, and Kircher (2004); Metcalfe et al. (2012).

33. Harvey (1985); Bentall, Baker, and Havers (1991); Garrison et al. (2017); Simons et al. (2010).

34. Eichner and Berna (2016); Moritz and Woodward (2007); Moritz et al. (2014).

Chapter 5: Avoiding Self-Awareness Failure

1. Alter and Oppenheimer (2006); Alter and Oppenheimer (2009); Reber and Schwarz (1999); Hu et al. (2015); Palser, Fotopoulou, and Kilner (2018); Thompson et al. (2013).

2. Kahneman (2012).

3. Schooler et al. (2011).

4. Smallwood and Schooler (2006).

5. Goldberg, Harel, and Malach (2006).

6. Reyes et al. (2015); Reyes et al. (2020).

7. Metzinger (2015).

8. Metcalfe and Finn (2008); Kornell and Metcalfe (2006); Rollwage et al. (2020); Koizumi, Maniscalco, and Lau (2015); Peters et al. (2017); Samaha, Switzky, and Postle (2019); Zylberberg, Barttfeld, and Sigman (2012).

Chapter 6: Learning to Learn

1. “Equipping People to Stay Ahead of Technological Change,” The Economist, January 14, 2017, www.economist.com/leaders/2017/01/14/equipping-people-to-stay-ahead-of-technological-change.

2. Christian Jarrett et al., “How to Study and Learn More Effectively,” August 29, 2018, in PsyCruch, produced by Christian Jarrett, podcast, 13:00, https://digest.bps.org.uk/2018/08/29/episode-13-how-to-study-and-learn-more-effectively/; Christian Jarrett, “All You Need to Know About the ’Learning Styles’ Myth, in Two Minutes,” Wired, January 5, 2015, www.wired.com/2015/01/need-know-learning-styles-myth-two-minutes/.

3. Knoll et al. (2017).

4. Ackerman and Goldsmith (2011).

5. Bjork, Dunlosky, and Kornell (2013); Kornell (2009); Kornell and Son (2009); Karpicke (2009); Zimmerman (1990).

6. Dunlosky and Thiede (1998); Metcalfe and Kornell (2003); Metcalfe and Kornell (2005); Metcalfe (2009).

7. Schellings et al. (2013); De Jager, Jansen, and Reezigt (2005); Jordano and Touron (2018); Michalsky, Mevarech, and Haibi (2009); Tauber and Rhodes (2010); Heyes et al. (2020).

8. Chen et al. (2017).

9. Diemand-Yauman, Oppenheimer, and Vaughan (2011); “Sans Forgetica,” RMIT University, https://sansforgetica.rmit.edu.au/.

10. In 2016 the penalty for guessing was removed, see Test Specifications for the Redesigned SAT (New York: College Board, 2015), 17—18, https://collegereadiness.collegeboard.org/pdf/test-specifications-redesigned-sat-1.pdf. Ironically, this change in the rules may have inadvertently removed the (also presumably unintended) selection for metacognition inherent to the previous scoring rule. See Higham (2007).

11. Koriat and Goldsmith (1996).

12. Bocanegra et al. (2019); Fandakova et al. (2017).

13. Bandura (1977); Cervone and Peake (1986); Cervone (1989); Weinberg, Gould, and Jackson (1979); Zacharopoulos et al. (2014).

14. Greven et al. (2009); Chamorro-Premuzic et al. (2010); Programme for International Student Assessment (2013).

15. Kay and Shipman (2014).

16. Clark and Chalmers (1998); Clark (2010); Risko and Gilbert (2016); Gilbert (2015); Bulley et al. (2020).

17. Hu, Luo, and Fleming (2019).

18. Ronfard and Corriveau (2016).

19. Csibra and Gergely (2009); Lockhart et al. (2016).

20. Bargh and Schul (1980); Eskreis-Winkler et al. (2019).

21. Trouche et al. (2016); Sperber and Mercier (2017).

22. Koriat and Ackerman (2010).

23. Clark (2010); Mueller and Oppenheimer (2014).

Chapter 7: Decisions About Decisions

1. Mark Lynas, interview with Dominic Lawson, February 4, 2015, in Why I Changed My Mind, produced by Martin Rosenbaum, podcast, 15:30, www.bbc.co.uk/sounds/play/b0510gvx.

2. Van der Plas, David, and Fleming (2019); Fleming (2016).

3. Fleming, van der Putten, and Daw (2018).

4. Rollwage et al. (2020).

5. Klayman (1995); Park et al. (2010); Sunstein et al. (2016); Kappes et al. (2020).

6. Rollwage et al. (2020); Talluri et al. (2018).

7. Rollwage and Fleming (in press).

8. Rollwage, Dolan, and Fleming (2018).

9. De Martino et al. (2013).

10. De Martino et al. (2013); Folke et al. (2016). Intriguingly, people’s eye movements also revealed the difficulty of the choice; they looked back and forth between the two options more often when they were uncertain. But only explicit confidence ratings predicted future changes of mind.

11. Frederick (2005); Evans and Stanovich (2013); Thompson et al. (2013); Ackerman and Thompson (2017).

12. Toplak, West, and Stanovich (2011); Pennycook, Fugelsang, and Koehler (2015); Pennycook and Rand (2019); Young and Shtulman (2020).

13. Johnson and Fowler (2011).

14. Anderson et al. (2012).

15. Hertz et al. (2017); Von Hippel and Trivers (2011).

16. Bang et al. (2017); Bang and Fleming (2018); Bang et al. (2020).

17. Edelson et al. (2018); Fleming and Bang (2018); Dalio (2017).

18. Amazon, “To Our Shareholders,” 2017, www.sec.gov/Archives/edgar/data/1018724/000119312518121161/d456916dex991.htm.

Chapter 8: Collaborating and Sharing

1. Shea et al. (2014); Frith (2012).

2. This is a version of the Bayesian algorithm that we encountered in Chapter 1 that combines two sensory modalities, such as seeing and hearing, except now the combination is being done across brains, rather than within the same brain. The model also suggests that the benefit of interaction breaks down if the observers are too dissimilar. This is indeed what was found: for dissimilar pairs, two heads are worse than the better one. Bahrami et al. (2010); Fusaroli et al. (2012); Bang et al. (2014); Koriat (2012).

3. Bang et al. (2017); Patel, Fleming, and Kilner (2012); Jiang and Pell (2017); Jiang, Gossack-Keenan, and Pell (2020); Goupil et al. (2020); Pulford et al. (2018).

4. Brewer and Burke (2002); Fox and Walters (1986).

5. Busey et al. (2000).

6. Wixted and Wells (2017).

7. National Research Council (2015).

8. Barney Thompson, “’Reasonable Prospect’ of Lawyer Being Vague on Case’s Chances,” Financial Times, November 25, 2018, www.ft.com/content/94cddbe8-ef31-11e8-8180-9cf212677a57; Robert Rothkopf, “Part 1: Miscommunication in Legal Advice,” Balance Legal Capital, November 23, 2018, www.balancelegalcapital.com/litigation-superforecasting-miscommunication/.

9. Tetlock and Gardner (2016).

10. Firestein (2012).

11. Open Science Collaboration (2015); Camerer et al. (2018).

12. Rohrer et al. (in press).

13. Fetterman and Sassenberg (2015).

14. Camerer et al. (2018).

15. Pennycook and Rand (2019).

16. Rollwage, Dolan, and Fleming (2018); Ortoleva and Snowberg (2015); Toner et al. (2013).

17. Schulz et al. (2020).

18. Fischer, Amelung, and Said (2019).

19. Leary et al. (2017).

20. Bang and Frith (2017); Tetlock and Gardner (2016).

Chapter 9: Explaining Ourselves

1. Cleeremans (2011); Norman and Shallice (1986).

2. Beilock and Carr (2001); Beilock et al. (2002).

3. Sian Beilock, “The Best Players Rarely Make the Best Coaches,” Psychology Today, August 16, 2010, www.psychologytoday.com/intl/blog/choke/201008/the-best-players-rarely-make-the-best-coaches; Steven Rynne and Chris Cushion, “Playing Is Not Coaching: Why So Many Sporting Greats Struggle as Coaches,” The Conversation, February 8, 2017, http://theconversation.com/playing-is-not-coaching-why-so-many-sporting-greats-struggle-as-coaches-71625.

4. Quoted in James McWilliams, “The Lucrative Art of Chicken Sexing,” Pacific Standard, September 8, 2018, https://psmag.com/magazine/the-lucrative-art-of-chicken-sexing. It is plausible that global metacognition remains intact in these cases. Despite having poor local insight into choice accuracy, the expert sexers presumably know they are experts.

5. Weiskrantz et al. (1974); Ro et al. (2004); Azzopardi and Cowey (1997); Schmid et al. (2010); Ajina et al. (2015). See Phillips (2020) for the view that blindsight is equivalent to degraded conscious vision, rather than qualitatively unconscious.

6. Kentridge and Heywood (2000); Persaud, McLeod, and Cowey (2007); Ko and Lau (2012).

7. Hall et al. (2010).

8. Johansson et al. (2005); Nisbett and Wilson (1977). The Lund experiments build on a famous paper published in 1977 by the social psychologists Richard Nisbett and Timothy Wilson. The article, “Telling More Than We Can Know: Verbal Reports on Mental Processes,” is one of the most cited in psychology. They surveyed literature suggesting that we have little insight into the mental processes that guide our behavior. This was in part based on experiments in which individuals would have a pronounced choice bias—for instance, choosing the rightmost item in a set of otherwise identical clothing—without any inkling that they were biased in such a fashion.

9. Gazzaniga (1998); Gazzaniga (2005).

10. Hirstein (2005); Benson et al. (1996); Stuss et al. (1978).

11. Wegner (2003).

12. Wegner and Wheatley (1999); Moore et al. (2009); Metcalfe et al. (2012); Wenke, Fleming, and Haggard (2010).

13. Vygotsky (1986); Fernyhough (2016).

14. Schechtman (1996); Walker (2012).

15. Frankfurt (1988); Dennett (1988); Moeller and Goldstein (2014).

16. Crockett et al. (2013); Bulley and Schacter (2020).

17. Pittampalli (2016).

18. Stephen M. Fleming, “Was It Really Me?,” Aeon, September 26, 2012, https://aeon.co/essays/will-neuroscience-overturn-the-criminal-law-system.

19. Case (2016); Keene et al. (2019).

Chapter 10: Self-Awareness in the Age of Machines

1. Hamilton, Cairns, and Cooper (1961).

2. Turing (1937); Domingos (2015); Tegmark (2017); Marcus and Davis (2019).

3. Braitenberg (1984).

4. Rosenblatt (1958); Rumelhart, Hinton, and Williams (1986).

5. Bengio (2009); Krizhevsky, Sutskever, and Hinton (2012); Schäfer and Zimmermann (2007).

6. Some caveats apply here. What does and does not count as an internal representation is a contentious issue in cognitive science. The philosopher Nicholas Shea has suggested that representations come into play when they help explain the functions of what the system is doing beyond simple, causal chains. For instance, we might say that part of a neural network or a brain region “represents” seeing my son’s face, because that representation is activated under many different conditions and many different inputs—it will become active regardless of whether I see his face from the side or from the front, or whether the lighting conditions are good or poor—and that such a representation is useful as it leads me to act in a particular, caring way toward him. In more technical terms, representations like these are “invariant,” or robust, across multiple conditions, which makes them a useful level of explanation in psychology and cognitive science (imagine trying to explain what face-selective neurons in a neural network were doing without talking about faces, and instead just talking about light and shade and inputs and outputs). Pitt (2018); Shea (2018).

7. Khaligh-Razavi and Kriegeskorte (2014); Kriegeskorte (2015); Güçlü and Van Gerven (2015).

8. Silver et al. (2017).

9. James (1950).

10. Marcus and Davis (2019).

11. Clark and Karmiloff-Smith (1993); Cleeremans (2014).

12. Yeung, Cohen, and Botvinick (2004).

13. Pasquali, Timmermans, and Cleeremans (2010). For other related examples, see Insabato et al. (2010) and Atiya et al. (2019).

14. Daftry et al. (2016); Dequaire et al. (2016); Gurău et al. (2018); Gal and Ghahramani (2016); Kendall and Gal (2017).

15. Clark and Karmiloff-Smith (1993); Wang et al. (2018).

16. Georgopoulos et al. (1982).

17. Hochberg et al. (2006).

18. Schurger et al. (2017).

19. Rouault et al. (2018).

20. Samek et al. (2019).

21. Harari (2018).

Chapter 11: Emulating Socrates

1. Maguire et al. (2000); Schneider et al. (2002); Zatorre, Fields, and Johansen-Berg (2012); Draganski et al. (2004); Woollett and Maguire (2011); Scholz et al. (2009); Lerch et al. (2011).

2. Harty et al. (2014); Shekhar and Rahnev (2018).

3. Hester et al. (2012); Hauser et al. (2017); Joensson et al. (2015).

4. Cortese et al. (2016); Cortese et al. (2017).

5. Carpenter et al. (2019).

6. Sinclair, Stanley, and Seli (2020); Max Rollwage, Philippa Watson, Raymond J. Dolan, and Stephen M. Fleming, “Reducing Confirmation Bias by Boosting Introspective Accuracy” (in prep).

7. Cosentino (2014); Leary (2007).

8. This intuition has received support in the lab. Careful experiments show that subjects can only remember and report a subset of several objects (such as letters or a bunch of oriented lines) that are briefly flashed on a screen. This itself is not surprising, because our ability to keep multiple things in mind at any one time is limited. Strikingly, however, subjects are able to accurately identify any individual object in the array if they are asked to do so immediately after the set of objects has disappeared. The implication is that all the objects in the array were consciously seen at the time of the stimulus, but that without being cued to report individual locations, they are quickly forgotten and so overflow our ability to report them. Sperling (1960); Landman, Spekreijse, and Lamme (2003); Block (2011); Bronfman et al. (2014).

9. Phillips (2018); Stazicker (2018); Cova, Gaillard, and Kammerer (2020).

10. Lau and Rosenthal (2011); Rosenthal (2005); Brown, Lau, and LeDoux (2019).

11. Lau and Passingham (2006); Michel and Morales (2020); Panagiotaropoulos (2012).

12. Del Cul et al. (2009); Fleming et al. (2014); Sahraie et al. (1997); Persaud et al. (2011).

13. Chalmers (2018); Graziano (2019).

14. Dehaene, Lau, and Kouider (2017); Schooler (2002); Winkielman and Schooler (2009).

15. LeDoux (2016); LeDoux and Brown (2017).

16. La Berge et al. (1981); Baird et al. (2018); Dresler et al. (2012); Voss et al. (2014); Filevich et al. (2015).

17. Baird et al. (2014); Fox et al. (2012).

18. Schmidt et al. (2019); Van Dam et al. (2018); Di Stefano et al. (2016).