In a less elitist vein, some modern critics of the mass media continue to deplore the decline of the age of verbal reasoning in the face of our present-day immersion in visual media from advertising to the computer screen. According to their scenario, the tyranny of the viewers' pictorial immersion results in uncontrolled emotional involvement with, - and the resultant lack of critical distance from - the pictorial message (Buddemeier 1993; 20).
Whatever foundations such warnings against the manipulative power of pictures may have, we can only focus on one of its aspects, namely the question whether the alleged manipulative power of pictorial messages could also derive from an inherent semiotic potential to lie, that is, the creation of untrue pictorial statements with the intent to deceive.
According to Umberto Eco's Theory of Semiotics, the question of whether phenomena can be used to convey a lie should be considered as crucial evidence of their sign nature. On the contrary, something that cannot be used to lie, should not be considered as an object of semiotic investigation. Eco (1976: 7) states these ideas in the following much quoted passage:
Semiotics is concerned with everything that can be taken as a sign. A sign is everything which can be taken as significantly substituting for something else. This something else does not necessarily have to exist or to actually be somewhere at the moment in which a sign stands in for it.Thus semiotics is in principle the discipline studying everything which can be used in order to lie. If something cannot be used to tell a lie, conversely it cannot be used to tell the truth; it cannot in fact be used 'to tell' at all. I think that the definition of a 'theory of the lie' should be taken as a pretty comprehensive program for a general semiotics.There is little doubt that pictures can refer to something that does not exist or has even never existed, but do such pictures therefore lie?Surrealism has given ample evidence of paintings referring to mere imaginary objects.Consider, for example, Salvadore Dali's Burning Giraffe (1935), which shows a strange woman with open drawers protruding from her legs. We are hardly inclined to call the painter of this work a liar, but even the category of truth, at least in the positivist sense does not seem applicable.
Although it is clear that pictures can refer to factual reality and to the unreal, the question whether they can convey a truth or a lie remains disputed.
What is the semiotic potential of pictures? Can they express ideas that correspond to verbal messages at all, as the proverbial saying which states that "Pictures can tell a thousand words", suggests, or is the semiotic potential of a picture inferior to the one of language insofar as a picture is necessarily vague and in principle unable to depict any truth about the world, as some logocentric semioticians claim? If pictures cannot tell the truth it should also be impossible to use them in order to convey a lie.
The question of truth or lie in pictures has a semantic, a syntactic, and a pragmatic aspect. From a semantic point of view, a true picture must be one which corresponds to the facts it depicts. From a syntactic point of view, it must be one which represents an object and conveys a predication about this object, and from the pragmatic point of view there must be an intention to deceive on the part of the addresser of the pictorial message.
Let us begin with the semantic dimension of our topic. Photographs seem to be prototype of visual messages which are true because they fulfil the semantic criterion of correspondence to the facts. Under certain circumstances, photographs are even recognized by the courts as documentary evidence, which may replace evidence by ocular inspection or by verbal testimony (Robert 1974; 17).
A pertinent example is the legal status of a passport photo as a document for establishing the real identity of the person presenting the passport to the authorities. From the legal point of view, truth, in the sense of correspondence between a signifier and its referential object, can thus be derived from photographic pictures.
Semiotically, the correspondence of the photographic signifier with the object it depicts is grounded in what Peirce described as the indexical and the iconic nature of photography. Photographs correspond to the depicted world by their iconic nature because, as Peirce (CP 2.281), puts it, "we know that they are in certain respects exactly like the objects they represent." In addition to this correspondence by similarity, photographs also correspond to reality by their contiguity with the depicted object at the moment of their production. There is a "physical connection" between the signifier and its referential object since, as Peirce (CP 2,281) argues "photographs have been produced under such circumstances that they were physically forced to correspond point by point to nature." By this relation of productive causality, the photographic picture is defined as an indexical sign.
It is primarily because of this indexical signature that we tend to see in the photographic signifier an affirmation of the existence of the depicted object. A semiotician who emphasized various aspects of this indexical nature of photography is Roland Barthes. In his words, the photograph is "an emanation of past reality" (Barthes 1980: 88), "one could think that photography always carries its referent with itself" (1980: 5), and the "noema of photography"is its message "this is the way it has been" (1980:77). Family photos, which remind us of real situations lived in the past, press photos, which document a historical event, such as the Stalin-Roosevelt-Churchill meeting of Teheran in 1943, or scientific photos, which show a real world object in all its details, are typical examples of indexical photographic reference and iconic correspondence between the photographic signifier and its object which testify to the truth potential of the photograph.
Nevertheless, everyone knows that photographic correspondence can be manipulated. The referential object may be transformed in the picture, and its viewers arrive at the illusive or deceptive impression of a nonexisting object. This deceptive potential of the medium was recognized early in the history of photography and made use of in techniques, such as retouch, colour filtering, solarization, double exposure.
By retouching, the signifier referring to an existing object could be made to disappear. By montage, a nonexisting object could make its appearance on the scene. Thus photography became a medium which lent itself to manipulation, deception, fakes, and forgeries. The more recent developments in computer graphics, with the new possibilities of shape blending, distortion, simulation, and other modes of digital image manipulation have greatly increased this deceptive potential of the medium.
Manipulations of the photographic image provide a rationale for Umberto Eco's (1984: 223) argument that "photographs can lie". However, instead of a lie, these are mere visual metaphor, hyperbols not to be taken seriously. The difference between a really deceptive fake, a genuine visual lie, and our topic is in the pragmatic dimension of the photographic message. From the semantic point of view, our examples do exemplify the pictorial potential of lying. Just like fakes, manipulated photos are visual messages which depict, but do not correspond to the reality depicted.
But before further specifying the pragmatic differences between lying and other modes of visual communication, we have to examine the syntactic dimension of truth and falsehood in pictorial messages.
In language, only sentences, and not individual words, can be true or false. The statement The cat is on the mat may be true or false, but not the individual words cat and mat. Truth values can only be derived from sentences or propositions in which a subject or argument is in a syntactic relation to a predicate. Is it possible to discover similar syntactic conjunctions of visual signs in pictures?
Since there are no words nor verbal propositions in pictures, let us use the more general semiotic terminology which Peirce introduced in the framework of his theory of signs: rheme, as the more general semiotic equivalent of words, and dicent, as the general equivalent of propositions. The question is then, can pictures function as autonomous dicentic signs, or do they only consist of rhematic signs? Do pictures only represent objects, or can they represent objects together with predications about these objects? For three very different reasons, the answers which the theory of pictorial representation has given to this question have been negative. These three arguments may be called contextual incompleteness, non-segmentability, and dicentic vagueness.
The argument of contextual incompleteness was first exposed by Gombrich (1960: 58-59). In his view, pictures alone can never function like true or false statements. Only when a picture is accompanied by a caption or label can the resulting text-picture message convey a true or false proposition. Captions below press photos or a name below the picture of an object are his examples. The logician Bennett (1974: 263) interprets the picture in such text-picture combinations as "predicates in schemes of predication". According to this view, the photo of a Siberian Husky above the caption "Siberian Husky" functions as the pictorial predicate which combines with the written name as its verbal argument to form a true verbal-pictorial statement. An example of a false message of this kind would be one of Rene Magritte's paintings of objects with deceiving labels, for example, his work La Table, I'Océan, le Fruit (1927),where the label "table" is attached to the picture of a green leaf and the label "fruit" to the picture of a jug. In such verbal-visual messages, it is not the picture alone which forms the proposition, and therefore Bennett (1974: 259) concludes: "Pictures are not themselves true or false, but only parts of things that can be true or false."
Muckenhaupt (1984:88), in his book Text and Picture, basically agrees with Bennett with respect to this general argument of contextual incompleteness of pictures, but believes that the image in the text-picture context does not function like a predicate, but rather like the argument of a proposition. According to this interpretation, the picture of a car in a police photo functions as the pictorial argument of a dicentic message whose predicates are verbally expressed in the numbers on the license plate and on the speedometer.
Whether the picture functions like an argument or as a predicate, what these interpretations have in common is that they consider the picture as an incomplete rhematic message which can function only as part of a larger dicentic whole when it appears in conjunction with a verbal message. Against this logocentric thesis of the dicentic incompleteness of pictures, I would like to argue that the function of pictures in text-picture combinations says nothing about the semiotic potential of pictures seen without labels or captions. The thesis that pictorial messages can only be completed by their verbal anchorage is rather an indicator of the logocentric bias to be found in the current theory of pictorial representation. In fact, although pictures without verbal anchorage may have become rare in our age of multimedia communication, such pictorial messages are by no means uncommon. In pictorial genres such a paintings, family photos, or touristic slides, the lack of verbal anchorage is even the rule.
Nevertheless, we still have to decide whether we can expect to find anything like the dicentic duality of verbal argument-predicate structures in such pictorial messages, and this brings us to the second argument against the assumption of the dicentic structure of pictures, the argument of non-segmentability. This argument is nicely developed in a paper by Jerry A. Fodor entitled "Imagistic Representation." Fodor (1981: 64-66) considers the possibility of a language, called, for the sake of argument, Iconic English, in which pictures might take the role that words play in a natural language. He concludes that no such pictorial language could exist because the linearization of arguments and predicates would prevent such pictorial words from being interpreted as a propositional whole. Fodor's example is: "Suppose that, in Iconic English, the word 'John' is replaced by a picture of John and the word 'green' is replaced by a green patch. Then the sentence 'John is green' comes out as (say) a picture of John followed by a green picture. But that doesn't look like John's being green; it doesn't much look like anything."
In his search for a pictorial equivalent to verbal propositions, Fodor commits the error of projecting the linearity of verbal language onto the visual domain where simultaneity is the structural principle relating the rhematic elements in question. Against Fodor's logocentric bias, we have to raise the question why the mere picture of green-skinned John should not suffice to derive the holistic pictorial propositional message "John is green"? Would not the photo of our green John testify to his unusual colour in an even much more convincing way than the verbal statement "John is green"? We claim that the argument "John" and the predicate "is green" must thus be sought in pictorial simultaneity and not in contiguity; or, if the linguistic analogy is preferred: the visual predicate is suprasegmental to the segmental visual argument.
The thesis of such a propositional structure in pictorial messages has actually been suggested earlier, namely in the semiotic theory of codes of the 1960's when the search for analogies between verbal and nonverbal messages was on the agenda of semiotic studies. Eco (1968: 236), e.g., following Prieto (1966), argues that pictures always have a propositional structure since "even the roughest silhouette of a horse does not correspond to the verbal sign 'horse', but to a series of possible propositions of the type 'standing horse in profile', 'the horse has four legs', 'this is a horse' etc." This early idea of a propositional structure in pictures, however, was not pursued very systematically since the discussion at the time was focused too much on the search for visual equivalents to the structural dividing line between the levels of first and second articulation in language, i.e., to words and phonemes.
Nowadays, in the era of cognitive approaches to pictorial perception, since new evidence for the interrelation between visual cognition and propositional coding in our mental representation of pictures has been found (cf., e.g., Jorna 1990), the topic deserves to be resumed and further explored.
After concluding that pictures do have the potential of fulfilling the criterion of propositional structure, let us turn to the third syntactic argument against the possibility of assigning truth values to pictures, the argument of dicentic vagueness. This argument claims that pictorial messages are so ambiguous, vague, and polysemous that they cannot serve to prove any truth or falseness. Both Gombrich and Fodor have defended this point of view.
As far as ambiguity is concerned, Wittgenstein(1953: 140b) is quoted as a witness, who once remarked that a man walking up a hill forward corresponds equally, and in the same way, to a man sliding down the hill backward. This may well be so, but there is ambiguity in language, too, which cannot testify against the truth potential of sentences either. Even the classical example of an unambiguous sentence, The cat is on the mat, may have an ambiguity to it since "being on the mat" is a slang metaphor which can make the sentence mean "The cat is in trouble". Furthermore, the same picture of Wittgenstein's man which may be ambiguous in one respect may well convey other truths about this man, e.g., facts about his face, figure, clothing or age.
Gombrich develops two arguments to prove that pictures cannot convey anything like a statement, and hence no true or false messages. The first argument is that pictures are vague, while sentences are not. Gombrich (1972: 82) explains:
The sentence from the prime: "The cat sits on the mat" is certainly not abstract, but although the primer may show a picture of a cat sitting on a mat, a moment's reflection will show that the picture is not the equivalent of the statement. We cannot express pictorially whether we mean "the" cat (an individual) or "a cat" (a member of a class).This argument is clearly logocentric. It does not ask whether pictures can convey statements, but asks whether it can convey the same statement as a given sentence. The answer would be different if the picture were the point of departure in the comparison with verbal statements. A particular photograph of a cat on a mat, being an indexical sign, is certainly in the first place about an individual cat and not about a member of a class. Furthermore, the sentence The cat is on the mat is in many respects much vaguer than a photo. While the hearer of the sentence has to rely on many supplementary pieces of knowledge in order to ascertain the truth value of the verbal statement - e.g., which cat? or which mat? - the viewers of the photo have many more visual signifiers at their disposal to ascertain the truth of this pictorial statement. The individuality of the cat and the mat an be easily identified in many details.
The logocentric bias behind Gombrich's argument is even clearer when he continues to discuss pictorial polysemy as a reason to contest the assertive potential of pictures. In his view, "although the sentence may be one possible description of the picture, there are an infinite number of other true descriptive statements you could make such as 'There is a cat seen from behind,' or for that matter 'There is no elephant on the mat' " (Gombrich 1972: 82).
Fodor (1981: 66-67) derives the same argument from a different example:
Suppose that the picture that corresponds to "John is fat" is a picture of John with a bulging tummy. But then, what picture are we going to assign to "John is tall"? The same picture? If so, the representational system does not distinguish the thought that John is tall from the thought that John is fat. (...) The trouble is precisely that icons are insufficiently abstract to be the vehicles of truth.Against Gombrich's and Fodor's view that pictorial polysemy prevents pictures from being vehicles of truth, I would like to argue that a message which conveys a plurality of facts about the world must not therefore be less true than a message that conveys only a single true statement. Neither polysemy nor ambiguity can thus be accepted as general arguments against the truth potential of pictures.
Incidentally, the degree of pictorial polysemy and ambiguity tends to be greatly overestimated. Just like the polysemy of language, the plurality of pictorial meanings is restricted by contextual, cotextual and cultural knowledge. It is therefore absurd to conclude, as Fodor (1981: 68) does, that the picture of fat John "corresponds equally to John's being pregnant since, if that is the way that John does look when he is fat, it is also, I suppose, the way that he would look if he were pregnant."
Turning to the pragmatic aspect of pictorial truth, we come back to the question whether pictures can assert at all. This question has to be considered because only those false messages count as a lie which are expressed in the assertive modality (cf. Kjerup 1974, 1978; Eaton1980; Korsmeyer 1985). Any lie implies a statement or assertion since the liar has the intention to deceive and pursues the goal to make the addressee believe in the truth of his or her proposition. Nothing can be judged as true of false if it is only expressed in the modality of possibility, fictionality, imagination, exemplification or as a mere question. This is why we could see no deceptive intention in the photo of our Man Without Qualities, in the manipulated photo of the buses in the soccer stadium, or in our distorted portrait. But can pictures assert at all? Is not their function restricted to the mere showing of the real or the imaginary?
At this point we have to consider one of the most serious arguments against the assertive potential of pictures, which is the argument of their pragmatic indeterminacy. It was Wittgenstein (1953: 22) who developed it with the following example:
Imagine a picture representing a boxer in a particular stance. Now, this picture can be used to tell someone how he should stand, should hold himself; or how he should not hold himself; or how a particular man did stand in such-and-such a place; and so on. One might (using the language of chemistry) call this picture a proposition-radical.According to Wittgenstein, the pragmatic function of pictures is thus open and undetermined. (Notice that Wittgenstein explicitly acknowledges the propositional potential of pictorial signs by calling it a propositional-radical.) This idea of pragmatic openness of pictures is one which Peirce ascribed in particular to the pictorial genre of photography, by defining photos as indexical signs. Insofar as they function as indices, photos are characterized by the criteria which Peirce (CP 3.361) specified for indexicality in general, namely: "The index asserts nothing; it only says 'There!' It takes hold of our eyes, as it were, and forcibly directs them to a particular object, and there it stops."
And yet, pictures are used for assertive purposes in situations which exclude other pragmatic functions. In language, the speech act of asserting is effected by means of a proposition which represents an actual state of affairs. Police photos and scientific illustrations are equally used to represent, and hence to assert, an actual state of affairs. Only because they assert, and not for any other pragmatic function, can they serve as legal or scientific documents of truth. The assertive potential is even inherent in the genre of photography. Only a photo, and not a painting of a crime will be accented as a document of truth in court.
Whenever signs can be used for asserting the truth they can also be used to deceive. If they assert they will be used as lies. A daily newspaper which publishes a photo in its news section asserts the reality of the scene in question. A manipulated photo of an honest politician shown in a scene toasting notorious gangsters whom he has never seen in reality (cf. Worth 1975: 100) is thus a photographic lie. Because of the documentary nature of this pictorial genre we take the photo as an assertion of the false scene as long as we are unaware of the manipulation. A painting of the same scene could only serve as a lie if accompanied by the statement of a witness testifying to its truth.
A final question to be considered is whether the assertive function of pictures can be derived from pictures alone or whether nonpictorial signs are required as indicators of their truth claim. The answer is that pictures and sentences in this respect, are both alike and different. They are alike because sentences in isolation cannot be judged for their truth either. The cat is on the mat is a sentence whose function may be to assert, but it may also be used with a poetic or a metalingual purpose because it rhymes or because it exemplifies a particular way of using language. Thus, both verbal and pictorial messages have to be interpreted within their larger context.
The difference between verbal and pictorial assertions is that the contextual indicators of an assertion in the medium of language can be expressed in the same medium, while those of pictorial messages cannot. While we can verbally reinforce the credibility of our claims by illocutionary verbs such as "I assert that", "I declare that", or "I swear that", and similar metalingual devices, pictures have no such metasemiotic means of asserting their truth (cf. Kjerup 1978: 65), unless the inherent assertive force of photographs is counted as such a metasemiotic device.
Notice, however, that the explicit contextual use of an illocutionary label of assertion in language is rather the exception than the rule (cf. Wittgenstein 1953: 22) and that there are also many contextual indicators of truth or lies which are nonverbal, e.g., the nonverbal reaction of blushing and similar reactions of interest in a lie detector test. Other nonlinguistic determinants of the assertive power of verbal utterances include the credibility of the witness or the situational probability of the truth of the verbal claim.
Questioning the truth value of pictures has had a long philosophical tradition. A logocentric bias against the truth potential of pictures can be found as early as with Plato, who wrote: "Painting is far from truth, and therefore, apparently, painting has the effect of reaching only little of everything, and that only in a shadow image" (Politeia X, 598b). The galactic evolution of pictures from Plato's shadow images to documentary and computer manipulated photographs has made a reconsideration of the topic necessary. Semiotics, although not immune against logocentrism, has provided tools for analyzing the topic of truth or lie in pictures without the logocentric bias. Sebeok (1986), e.g., has shown that lying is by no means restricted to verbal semiosis since it can be found with animals, and the semiotic of pictures has made advances in investigating pictures as a system of signs autonomous with respect of verbal language. The result of our study was that pictures can be used to assert or to deceive about facts from the semantic, syntactic, and with certain reserves, also from the pragmatic dimension. This does not mean that asserting and lying are very typical modes of pictorial information. Most of the manipulative strategies of pictorial information in the media are not direct falsifications of reality expressed in the assertive mood, but manipulations by means of a plurality of indirect modes of conveying meanings.
Bennett, John G. "Depiction and convention." In The Monist 58, pp255-268, 1974.
Buddemeier, Heinz. Leben in kunstlichen welten: Cyberspace, Videoclips und das tagliche Fernsehen. Stuttgart: Urachhaus, 1993.
Eaton, Marcia. "Truth in pictures." Journal of aesthetics and art criticism 39,15-26, 1980.
Eco, Umberto. (La struttura assente, trans.) Einfuhrung in die Semiotik. Muchen: Fink, (1968) 1972.
--- A theory of semiotics. Bloomington: Indiana University Press, 1976.
--- Semiotics and the philosophy of language. Bloomington: Indiana University Press, 1984.
Fodor, Jerry A. "Imagistic representation." In: Ned Block, ed. Imagery. Cambridge, MA: MIT Press, 63-86, 1981.
Gombrich. Ernst H. Art and illusion. London: Phaidon, (1960) 1968.
---Symbolic images. Edinburgh: Phaidon, (1972) 1975.
Jorna, Rene J. Knowledge representation and symbols in the mind. Tubingen: Stauffenburg, 1990.
Kjerup, Seren. "Doing things with pictures." In The Monist 2, 216-235, 1974.
--- "Pictorial speech acts." Erkenntnis 12 55-71, 1978.
Korsmeyer, Carolyn. "Pictorial assertion." Journal of Aesthetics and Art Criticism 43 257-265, 1985.
LeBon, Gustave. Psychologie des foules, dt. Psychologie der Massen. Stuttgart: Kroner, (1895) 1973.
Muckenhaupt, Manfred. Text and Bild. Tugingen: Narr, 1986.
Peirce, Charles S. Collected papers. Vols. 1-6 ed. Hartshorne, Charles, and Weiss, Paul; vols. 7-8 ed. Burks, Arthur W. Cambridge, Mass." Harvard University Press, 1931-58.
Prieto, Luis J. Messages et signaux. Paris: Presses Universitaires, 1966.
Robert, Oliver. Der Augenschein im Strafprozelss. Zurich: Schulthess, 1974.
Searle, John R. Speech acts. Cambridge: University Press, 1969.
Sebeok, Thomas A. "Can animals lie?" In T.A. Sebeok I think I am a verb. New York: Plenum, 126-130, 1986.
Wahner, Matthias. Mann ohne Eigenschaften - Man without qualities. Munchen: Stadtmuseum, 1994.
Wittgenstein, Ludwig. Philosophische Untersuchungen - Philosophical Investigations. Oxford: Blackwell, 1953.
Worth, Sol. "Pictures can't say ain't." Versus 12,85-108, 1975.
Winfried Soth is Professor of English Linguistics at the University of Kassel, Germany. He is the author of several books and numerous articles in linguistics and the semiotics of art, literature and the media. His Handbook of Semiotics appeared in English in 1990 (Indiana University Press).