Go to Semiotic Review of Books Home Page
Go to SRB Highlights
Go to SRB Archives


SRB Archives

This editorial appeared in Volume 12(1) of The Semiotic Review of Books.

Symbols In Your Face

Symbols In Your Face


Andrew Young (1998) Face & Mind. Oxford: Oxford University Press.


By Chris Westbury


“It is of the essence of symbols to be symbolic.”

Jacques Vaché, in a letter to André Breton



“Nothing complicated enough to be really interesting could have an essence.”

Dan Dennett, Darwin’s Dangerous Idea


Essentialism - the idea that the apparent categories of the world are grounded by corresponding essences - has suffered major blows in modern times. The rise of statistical thinking in the last century has given us the intellectual tools to conceive of a world without essences; a world decentered, with no controller overseeing it and no place where everything comes together. Scientific materialists today see no grand plan; no hierarchy of being; no Laplacean certainty. Everything is flux; everything moves; everything varies. Nothing is what it appears to be. Every apparent phenomenon dissolves into multitudinous microphenomena.

                When conceived of nearly half a century ago, cognitive science seemed to spring in part from an instinct to resist this anti-essentialist trend. With its emphasis on Von Neumannesque computations and well-specified algorithms, early cognitive science might be aptly characterized as a last-ditch effort to save an essence which defined mankind: rationality, intentionality, or symbolization. These were widely considered to be non-decomposable primitives, the fundamental building blocks of human computation. However, these and other functional primitives have now also succumbed (or are now succumbing) to the anti-essentialist juggernaut. A new anti-essentialist vision for cognitive psychology (outlined in Barkow, Cosmides & Tooby 1992; Bruner 1990; Elman et al 1996; Hendriks-Jansen 1996; Clark 1997; Pinker 1997, and presaged in Ryle 1949; Hinde 1957; Bateson 1972, 1979; Lorenz 1973; Wittgenstein 1958; see also Hallett 1991) emphasizes dynamic, microanalytic descriptions of apparent high-level primitives. Such a view is receiving growing empirical support from many recent developments, ranging from the generalization of evolutionary theory in the cognitive domain to a re-appraisal of the requirements and role of computationalism in light of the limited success of classical “symbol-manipulation” methods of artificial intelligence. Cognitive psychologists Kahneman and Tversky have helped usher in the change by showing that the notion of an idealized human rationality is a myth. (Kahneman, Slovic, & Tversky 1982; Piatelli-Palmarini 1994) They have recently been joined by neuroscientists (Damasio 1994) alleging the same thing from an independent neurological perspective. The philosopher Dan Dennett (1987) launched a concerted wave of anti-essentialist attacks against Brentano’s (1874) notion of intentionality, scandalizing old guard philosophers by suggesting that even a lowly artifact like a thermostat might be considered as an intentional system, if there were compelling reason to regard it so.

Work on the evolution of symbols (Hoffmeyer 1993; Lock & Peters 1999), the necessary embodiment of language, (Lakoff & Johnson 1999) and the development of communicative systems in nonhuman animals (Hauser 1996) all continue the same kind of functional deconstruction for symbolization. A young generation of cognitive scientists has no difficulty in accepting the notion that symbols do not refer (as early analytical philosophers implied) by their very essence, in a single focused beam of logically-consistent directed reference. They only appear to do so in the normal case because they are subject to a myriad of independently dissociable functional constraints. As has happened so often, what once appeared as a unity is now beginning to reveal a world of fine-grained variation.

                Those who study humans with brain damage recognize this way of thinking about symbolic reference to be consonant with a great deal of solid empirical data in neuropsychology. Brain damaged patients have been systematically studied for a century and a half, and have presented a myriad of well-documented functional dissociations of linguistic symbol access and use. Any textbook on neurolinguistics will include a list of symptom clusters around linguistic comprehension and production that leave no doubt that the two processes of comprehension and production of linguistic symbols cannot be neurological primitives. Each breaks down into a bewildering number of subfunctions independently computing such aspects as emotional tone, relevance to pragmatic practice, and similarity to other words on orthographic, phonological and semantic dimensions.

                Although analogous dissociations have been found for nonlinguistic symbolic domains, few have been as thoroughly studied as language. Andrew Young’s book, Face And Mind, presents a thorough overview of functional dissociations impacting on a symbolic domain in which normal humans operate so naturally that we have a hard time even conceiving of it as symbolic: face recognition. In opening up this domain, Young provides insight into just how deeply immersed humans are in the world of symbols, and provides fascinating insight into the strange and roundabout ways they are computed by our brains.

                What makes face recognition a fundamentally symbolic activity? Almost all semiotic theories follow De Saussure (1966 [1915]) in underscoring the importance of the arbitrariness of the sign. As Young points out in several places in his book, the relation between the surface form of face, and the identity or group membership of the person to whom that face belongs, is relatively arbitrary in just this sense, compared to many other types of visual objects. “For example, it would be possible to distinguish animals from items of furniture even in the case of unfamiliar members of each category, whereas reliable discrimination of unfamiliar politicians from unfamiliar stockbrokers could not be achieved on a purely visual basis.” (103) The nature of the relation between a face and the proper name of its owner is, of course, even more arbitrary than that between face and other measures of the semantics of identity. It is often easier to recognize a movie star as a movie star than it is to give his/her name.

The cognitive functions computing these two mappings - between a face and the semantics of identity, and from such semantic information to a name - can be independently damaged in humans. In the most well known face recognition disorder, prosopagnosia, a patient loses the ability to make the first relation. Such patients are no longer able to recognize faces at all, either by name or by semantic category (i.e., by recognizing a movie star as a movie star, even if unable to name who the star is). A loss of access to names – anomia - has also been reported (McKenna & Warrington 1980) which was limited to proper names.

The complexity of the mapping from faces to understanding becomes clearer when we appreciate how many other independent relevant sub-functions have been documented, all of which are discussed in Young’s book. Humans, for example, have specialized and independent neural systems for analyzing gaze direction; lip and tongue movements; distinctiveness of facial features; and facial expressions and their implications for emotional states. We are not only sensitive to familiarity independent of recognition, but familiarity is itself dissociable into conscious and unconscious components. We are independently sensitive to different face regions for different purposes: for example, we place far more emphasis on the eyes for determining familiarity, and on the nose for determining sex. Pigmentation is heavily used in recognition, and removing it while leaving all other identifying information intact makes faces very much harder to identify. We use different neural pathways to process familiar and non-familiar faces (just as we do for familiar and unfamiliar words). Face (and other object recognition) is modulated by maturational changes: studies have shown that face recognition performance by women undergoing puberty is worse than their performance before or after puberty. Recognition of even highly familiar people is sensitive to the context in which that person is encountered.

                Although some of these aspects of recognition are unique to faces, there is good reason for believing that understanding face recognition can shed light on recognition more generally. There is no evidence of a single region of the brain specifically devoted to human face recognition, although neurons specifically sensitive to facial orientation have been identified by single cell recording in nonhuman primates. Clinical data support the conclusion that face recognition relies upon non-specialized recognition functions. Prosopagnosics often show difficulties in recognizing or distinguishing other classes of objects. Young  lists cases of prosopagnosic patients who also had trouble distinguishing food types, animals (one ornithologist lost her ability to recognize bird species), automobiles, cards, coins, and abstract nonlinguistic symbols (such as dollar signs), especially when the exemplars within each semantic class were similar in form. Some prosopagnosics lose their ability to recognize individual nonhuman animals, but others do not. One symmetrical finding seems to defy the odds of finding cow farmers with brain damage: one prosopagnosic cow farmer has been studied who was still able to distinguish his individual cows following his brain injury, while another prosopagnosic cow farmer has been (independently) studied who was not.

                In light of the likelihood that findings related to face recognition have general applications to symbolic decoding, Young’s book has wide implications. Some of the most fascinating reading in the book suggests one particularly important general implication: that human symbolic reference must be in large part modulated by limbic input or, in lay terms, by emotions. The role of emotion in human cognitive functioning has become increasingly appreciated by cognitive neuroscientists in recent years, due to growing evidence that damage to pathways carrying emotional information from subcortical regions to the cortex can result in clear cognitive deficits. (Damasio 1994) Young’s book includes chapters on some colorful deficits that are relevant to our understanding of the role of emotions on cognitive functions. He describes a series of three related delusional misidentification syndromes known as Capgras Syndrome, Frégoli syndrome, and intermetamorphosis.

The defining symptom of Capgras syndrome (first defined in 1923 by Capgras and Reboul-Lachaux) is a belief by the diagnosed patient that familiar people have been replaced by identical doubles (impostors). Capgras patients have absolutely no insight into the implausible or delusional nature of their belief. They are quite convinced of it. Over 130 cases of the syndrome - including one (Ramachandran 1998) in which the object of the delusion was a favored poodle - have been reported since the initial report. Some of these have been very carefully studied, with results I will discuss below after briefly outlining the two related syndromes.

Frégoli’s syndrome was first described in 1927 by Courbon and Fail. It is named after an Italian actor known for his skill in impersonating others.  The defining symptom is that the affected patient misidentifies strangers as a familiar (usually famous) person. They do not mistake the appearance of the stranger, but attribute the apparent non-resemblance to the identified person to a disguise. This “hyper-identification” is, quite understandably, usually accompanied by feelings of being persecuted or hounded by the misidentified famous person. As Young (236) points out, this kind of hyper-identification may be considered as the inverse of the hypo-identification seen in Capgras syndrome.

Intermetamorphosis is a much rarer misidentification syndrome, first described in 1932 by Courbon and Tuques, which is characterized by both false recognition of identity (as in Frégoli syndrome) and of associated appearance. Patients with the syndrome misidentify strangers, not because they assume they are disguised, but because the strangers actually appear transformed in their appearance.

Although some cases (especially of intermetamorphosis) may be secondary to psychiatric disorders such as schizophrenia, the current consensus is that all three of these syndromes have a genuinely neuroanatomical component, insofar as they are caused by brain lesions. This suggests (under a key foundation assumption of cognitive neuropsychology) that their symptoms can shed light on what kind of computations a normal uninjured brain must be carrying out. Capgras patients have been the most carefully studied. The evidence in favour of a purely functional etiology for that syndrome is very strong. The exact locus of the necessary injury is still under dispute. However, a recent study of a Capgras patient (Ramachandran 1998) has shown that the functional effect is to cut off cortical face recognition areas from subcortical (emotional) input. Ramachandran’s patient showed none of the change in skin conductance (a sign of emotional activation) to familiar faces that accompany familiar face recognition in normals. Ramachandran has speculated (though it has not yet been demonstrated) that Frégoli syndrome may be secondary to an excess of limbic input into areas associated with face and object recognition.

One might predict from the hypothesis of limbic disconnection in Capgras syndrome and the claim that face recognition uses non-specialized functions that there should also exist a syndrome in which patients believe that other emotionally-salient familiar objects have been replaced with copies. Sure enough, such a syndrome exists and is discussed by Young. He reports on one subject with Capgras syndrome who also believed that pictures of famous buildings he was shown were actually pictures of copies of those buildings and on another case in which a patient insisted that familiar places had been duplicated. (Alexander et al 1979)

If we accept the position outlined above that face recognition (in particular) is aptly characterized as a form of symbolic decoding, then these findings are of more than passing semiotic interest. When we change the terminology from the neuropsychological to the semiotic, we can characterize the error in the delusional misidentification syndromes as being a systematic error in correctly decoding the referent of a sign. My wife’s face, for example, may be construed as a sign that bears only an arbitrary and conventional relationship to its referent, the woman who married me. There is no direct functional, causal, or semiotic connection between my wife’s facial features and her role as my wife: that is, no way to tell just from examining her face that she is married to me. If, in accordance with the limbic input hypothesis, we sever the input between limbic areas and the cortical area just behind my ears (the inferior temporal lobe) which is devoted to object and face recognition, then the face of my wife no longer refers (so far as I am concerned) to my wife at all, but to a copy of her. By an act of surgery, we change the apparent referent of a conventionally referring symbol.

We should not, of course, make too much of what is, after all, nothing more than an error. If I mistakenly tell you that the word cat refers to a dog, I do no thereby contribute useful data to semiotic theory. I have simply made an error. Similarly, if I insistently mistake my wife for a copy of my wife due to brain damage, I have simply made an error. However, if we wish to develop a biologically based understanding of symbolization - an understanding which rejects disembodied essentialist accounts of reference and insists on the central importance of the biological system underlying any interpretation (Hoffmeyer 1993) - then we need to take such neurologically based referential errors seriously. They are the main data that allow us to see how the apparently monolithic function of symbolic reference comes to be experienced by symbol users as monolithic. They reveal the kinds of blindness to which that illusionary phenomenological essentialism may subject us. They teach what we are leaving out when speaking of symbolism with a single word.

Young’s book is not primarily concerned with these matters. His goal is to outline what his work as a cognitive neuropsychologist has been able to discover about face recognition and, in the final chapters, to consider what implications the study of face recognition might have for understanding the bigger questions of his field. The book is composed of a series of reprinted articles, most of which originally appeared in scientific journals, or as chapters in specialized collections. For this reason, the book is directed primarily at cognitive psychologists. The neurological discussion is relatively thin. Many of the chapters report on psychological experiments with prosopagnosic subjects. Readers with no familiarity of experimental design or the methods of experimental psychology may not find these an easy read. Readers with the required background will find that Young writes in an engaging manner that is easy to follow and with an enthusiasm for his work that is infectious. However, they may be frustrated, as I was, that Young has done little to edit the chapters for the current collection. The chapters are quite repetitive, especially in their reproduction of Young’s own model of face recognition, which is explained and presented in diagram form numerous times. Although the chapters are ordered to form a rough conceptual continuum, no apparent work has been done to enhance the continuity given by ordering. Each chapter forms a unit unto itself, with a separate reference section rather than the more desirable single reference section at the back of the book.

                Despite these weaknesses, the book does recommend itself to anyone interested in understanding contemporary cognitive neuropsychological approaches to the human mind. It serves as an admirable overview of one well-conducted model research program, and demonstrates the conceptual riches that a little ingenuity and experimental effort can uncover. In doing so, it emphasizes what is becoming increasingly clear in scientific cognitive studies: that the human brain is not a Von Neumann computer, running through well-defined algorithms to reach certain conclusions. Rather it is, like all complex biological systems, a hodge-podge of quasi-independent sub-systems, which normally are mutually constraining. The mutual constraints enable the system to arrive, under normal circumstances, at answers that are good enough for most purposes - or, anyway, good enough for evolution’s purpose.

Such a hodge-podge of functionality leaves no place for essentialist theories of cognitive function. In particular, the essentialist notion that the human mind can be understood from the inside-out, as a symbol-processing computer, makes no sense when we understand that what are traditionally thought of as symbols must be the product, rather than simply the contents, of human brain processes. If humans are indeed a composite of quasi-independent neural sub-systems, we are not blessed with a laser sharp intentional system that latches on to the real referent in a real world of real signs. Our brains cobble together what they may, and deduce what they can about what there is from what they’ve got. From a multitude of limited dynamic building blocks acting together in probabilistic concert has sprung every apparent defining essence of humankind, including our astonishing ability to leap from a singular arbitrary construction to a universal concept: in short, to symbolize.


Chris Westbury is Assistant Professor in the Department of Psychology, University of Alberta, specializing in the area of behavioral and cognitive neuroscience.



Alexander, M.P., Stuss, D.T., and Benson, D.F (1979) “Capgras syndrome: a reduplicative phenomenon,” Neurology 29: 334-9.


Barkow, J.H., Cosmides, L., & Tooby, J. (1992) The Adapted Mind: Evolutionary Psychology and The Generation Of Culture. Oxford: Oxford University Press.


Bateson, G. (1972) Steps To an Ecology Of Mind. New York: Ballantine Books.


Bateson, G. (1979) Mind and Nature: A Necessary Unity. New York: Bantam Books.


Brentano, F. (1874) Psychologie vom empirischen Standpunkte. Leipzig.


Bruner, J. (1990) Acts Of Meaning. Cambridge, MA: Harvard University Press.


Capgras, J. & Reboul-Lachaux, J. (1923) “L’illusion des ‘sosies’ dans un délire systématise chronique,” Bulletin de la Société Clinique de Médecine Mentale 2: 6-16.


Clark, A. (1997) Being There: Putting Brain, Body, And World Back Together Again. Cambridge, MA: MIT Press.


Courbon, P. and Tusques, G. (1932) “Illusions d’intermetamorphose et de charme,” Annales Médico-Psychologiques 14:401-6.


Courbon, P. and Fail, G. (1927) “Syndrome d’illusion de Frégoli et schizophrénie,” Bulletin de la Société Clinique de Médecine Mentale 15: 121-5.


Damasio, A. R. (1994) Descarte’s Error: Emotion, Reason, and The Human Brain. New York: G.P Putnam’s Sons.


Dennett, D. (1987) The Intentional Stance.

Cambridge, MA: MIT Press.


Dennett, D. (1995) Darwin’s Dangerous Idea: Evolution And The Meanings Of Life. New York: Simon & Schuster.


Elman, J.L., Bates, E.A., Johnson, M.H., Karmiloff-Smith, A., Parisi, D., Plunkett, K. (1996) Rethinking Innateness: A Connectionist Perspective On Development. Cambridge, MA: MIT Press.


Hallett, G. L. (1991) Essentialism: A Wittgensteinian Critique. Albany: State University of New York Press.


Hauser, M. D. (1996) The Evolution of Communication. Cambridge, MA: MIT Press.


Hendriks-Jansen, H. (1996) Catching Ourselves In The Act: Situated Activity, Interactive Emergence, Evolution, and Human Thought. Cambridge, MA: MIT Press.


Hinde, R.A. (1957) “Consequences And Goals: Some issues raised by Dr Kortlandt’s paper on aspects and prospects of the concept of instinct,” British Journal Of Animal Behavior 5: 116-118.


Kahneman, D., Slovic, P., & Tversky, A., (Eds.)  (1982) Judgment Under Uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press.


Lock, A. & Peters, C.R. (1999) The Handbook of Human Symbolic Evolution. Malden, MA: Blackwell Publishers Inc.


Lorenz, K. (1977) Behind The Mirror: A Search for a Natural History of Human Knowledge. New York: Methuen & Company Ltd.


McKenna.P., & Warrington, E.K. (1980) “Testing for nominal dysphasia,” Journal of Neurology, Neurosurgery, & Psychiatry 43: 781-8.


Piatelli-Palmarini, M. (1994) Inevitable Illusions: How Mistakes of Reason Rule our Minds. New York: John Wiley and Sons.


Pinker, S. (1997) How the Mind Works. New York: W.W. Norton & Co.


Ramachandran, V.S & Blakeslee, S. (1998) Phantoms in the Brain: Probing the Mysteries of the Human Mind. New York: William Morrow & Company.


Ryle, G. (1949)  The Concept of Mind. London: Penguin.


Saussure, F. de (1972/1915) Course in general linguistics. Lasalle, Ill.: Open Court.


Wittgenstein, L. (1958) Philosophical Investigations. Oxford: Blackwell

Go to Semiotic Review of Books Home Page
Go to SRB Highlights
Go to SRB Archives