Picturebooks and Emotional Literacy

Although representation of emotions in literature is a well‐researched area, as is the study of young and adult readers’ affective response to literature, the rapidly expanding area of cognitive literary criticism builds on research in cognitive science to inform studies of readers’ cognitive and emotional engagement with literary texts (e.g., Hogan, 2011; Stockwell, 2002; Vermeule, 2010; Zunshine, 2006). So far there is little research within cognitive criticism that takes into consideration young readers, who not only lack the experience of a full range of human emotions, but also who have not yet fully developed their theory of mind and empathy skills.

Theory of mind, a concept from cognitive psychology, refers to the capacity to understand how other individuals think. Empathy refers more specifically to the capacity to understand how other people feel. It is intriguing for a picturebook theorist to consider how engagement with picturebooks can endorse theory of mind, which is an indispensable social skill (see Nikolajeva, 2012). It is equally intriguing to imagine how such purely speculative scholarship can inform both empirical research with young children and teachers’ classroom activities.

Young children have limited life experience of emotions, whereas picturebooks offer vicarious emotional experience that children can partake of. Fiction, as cognitive criticism claims, creates situations in which emotions are simulated. Reading picturebooks prepares children for dealing with empathy and mind‐reading in real life. Vast empirical research confirms that even very young children understand and respond to emotional dimension in picturebooks (see, e.g., Arizpe & Styles, 2003; Evans, 2009; Sipe & Pantaleo, 2008).

The visual response method (children drawing instead of responding verbally) circumvents young informants’ lack of verbal skills to articulate their comprehension. Close observations during experiments, including recording of children’s facial expression, body language, gaze movement, the duration of time spent on each spread, allows for reliable accounts of their engagement with multimodal texts. Limited experimental research, registering brain reactions, has investigated children’s affective responses to single — and typically simple — emotionally charged images and simple verbal descriptions of mental states.

However, what is accessed through such experiments are emotions triggered in a subject’s brain by the image itself. Thus a child feels joy seeing images of a smiling face, a bright sunlit landscape, or a cute animal, and they experience fear seeing a scary monster. Such responses are not substantially different from responses to real‐life experience, even though a visual representation of a monster is not exactly the same as a real‐life monster.

Direct affective responses are hard‐wired in the brain and evolutionarily conditioned. Against common sense, most of us have indeed laughed at funny images or become upset about disturbing ones. Cognitive criticism explains this by the complex mechanism of mirror neurons that enables our brain to simulate responses to representations as if they were real visual stimuli (see, e.g., Blackmore, 2005; O’Shea, 2005). Although a picture of Tyrannosaurus Rex presents no danger to the viewer, the brain still responds to the image as if it were real, because it has been evolutionarily trained to alert to possible danger.

Moreover, very young children may not yet perceive a representation as different from its referent in real life. Films such as Jurassic Park are built on the premise that representations of scary images affect the viewer just as if they were real. Educators frequently claim that certain picturebook images can be scary for children, whereas literary scholars tend to dismiss such allegations far too easily, ignoring the fact that the brain actually makes little distinction. A young child who smiles seeing a happy face in a picturebook or starts moving at the sight of a fictional character running and jumping, who cries over a picture of a dead animal or shudders at gaping jaws of a dinosaur, is truly experiencing the emotions as if they were real. In fact, they are real. This is something that as educators we need to understand and respect.

The same mechanism allows us to engage vicariously with fictive character’s emotions. In readers’ involvement with picturebooks, these two kinds of emotions, textual and extratextual, inevitably interact. Readers can be directly frightened by images of creatures that they believe are dangerous, such as dinosaurs, bears, or wolves. They can also be frightened by certain settings, such as dark woods. The danger, however, threatens fictional characters rather than readers. As soon as the reader acknowledges that the situation is fictional, the emotional engagement is shifted onto the characters.

Consequently, we should be able to read the character’s fear even if we do not experience the same emotion when looking at the image or understand that the character is sad without feeling distressed ourselves. We should be able to evaluate the character’s fear as ungrounded (for instance, that there is no monster under the bed) or predict that sadness will eventually be replaced by joy. It may sound simple, even self‐evident, and in everyday practice of engaging with fiction, we do this more or less automatically. Yet let us consider the implications of empathic engagement.

The development of empathy typically starts with recognizing basic emotions, such as joy, distress, fear, and anger (see, e.g., Evans, 2001). A vast number of picturebooks are focused on these emotions, treating them in a range from simple, literal, and straightforward to elaborate and metaphorical.

For instance, books as dissimilar in style and psychological depth as Frog Is Sad by Max Velthuijs and The Red Tree by Shaun Tan present the same narrative: The character is distressed, but becomes glad again. Basic emotions are universal and independent of verbal language. Physical manifestation of basic emotions, notably facial expression, but also body posture and gestures, normally do not require any special training. Young children may not know exactly what the verbal phrases “He was sad” or “She was frightened” mean, yet they will presumably respond to the visual representation of sadness or fear.

There is a wide scope of emotions that images, alone or in combination with words, can convey. Where the Wild Things Are portrays anger; Outside Over There portrays fear. Our emotional response to emotionally charged images is possible because we have stored (albeit inaccurate and fragmented) memories of the represented emotion, either from real‐life experience or from an earlier experience of fiction, whether verbal, visual, or multimedial. We may not have a direct experience of extreme distress or extreme fear, but the little experience we have is sufficient to trigger the memory.

When reading images, we are looking for recognizable external tokens of emotions, because this is how we use theory of mind in real life. Cognitive studies claim that the most prominent features that reflect human emotions are the eyes and the mouth and that these features are universal. This is probably why emoticons work for universal visual communication.

The shape of the mouth is arguably the most salient trait and is therefore used excessively in picturebooks. The absence of mouth, for instance, in The Red Tree or in Oliver Jeffers’s Lost and Found makes it immediately more difficult to read the character’s emotions. Happiness and sadness are otherwise clearly recognizable by the upturned and downturned corners of the mouth. A wide‐open mouth may signify fear or anger. Wide‐open eyes signal either surprise or fear. A combination of closed eyes and open mouth suggests distress. Raised eyebrows may convey anger.

Facial expression is, however, not the only way of communicating emotions. Cognitive science emphasizes that emotions are embodied, that is, connected to body movements and spatial position. Indeed, we can recognize emotions, in real life as well as in visual representations, through body language. Thus outstretched arms signal joy, whereas limp arms hanging on the sides of the body signal distress. In reading images, we project our own embodied emotions onto represented figures. Conversely, picturebooks offer images of characters that help us understand people’s emotions in real life.

It is hard to read an emotion when there is no clear external expression of it, for instance, in an animal or an inanimate object, because we have no real‐life experience of animals’ or objects’ feelings. Yet, paradoxically, picturebooks abound in nonhuman characters, including animals, toys, machines, objects, natural phenomena, and even blobs of color, as in Leo Lionni’s Little Blue and Little Yellow. We are able to engage with such characters’ emotions because we anthropomorphize them, that is, ascribe them human consciousness. As cognitive criticism claims, anthropomorphization is our way of making sense of the world.

For instance, in the opening page of H.A. Rey’s Curious George, we instantly recognize the character’s emotion as happiness, even without the prompt from the words. In real life, monkeys do not smile, and when they bare their teeth, it is typically a sign of aggression. Yet, through anthropomorphization, we engage with animal characters’ emotions as if they were humans. We can empathize with characters whom we in real life might find disgusting (frogs, caterpillars, spiders, mice, pigs) or dangerous (bears, lions, hippos, crocodiles). Likewise, we read emotions of nonhuman characters through their body language, even if they have no faces, for instance, in Shaun Tan’s The Lost Thing.

In addition, when reading picturebook images, we also apply our knowledge of artistic conventions. We read characters foregrounded and placed in the center of an image as happy and content, whereas we read characters crammed in corners or pushed into the background as unhappy, lonely, and scared. We attribute happiness to characters positioned high on the page. We connect certain moods with certain colors: red with aggression, yellow and green with joy, gray and black with distress, and brown perhaps with disgust because it reminds us of feces.

Not least, we may be familiar with graphic conventions such as motion lines, thought balloons, and emoticons. Motions lines, for instance, apart from conveying movement, may signify strong emotions such as fear and anger. Thought balloons can contain further images reflecting the character’s state of mind. They can also include images of pleasant, sad, or scary memories. The convention of emoticons makes a wide register of emotions easily understandable when we encounter them in multimedial narratives. Although it does not happen in real life, by convention, a zigzagged mouth implies anger or fear. Hair on end suggests extreme fear. Understanding of such conventions is a matter of visual literacy, and it can therefore be encouraged and trained.

More important, emotionally charged images in picturebooks are in most cases complemented by verbal statements, simple or complex, literal or metaphorical: “He was sad,” “She was frightened,” “He froze in horror,” “She boiled with anger.” Picturebook titles frequently offer a straightforward label for the emotion dealt with in the book: Frog Is Sad and Frog Is Frightened, by Max Velthuijs, or My Friend Is Sad by Mo Willems. A title may also be less literal; for instance, Velthuijs’s Frog Is Frog implies “Frog is happy.”

Images and verbal statements may be mutually complementary or enhancing; they can even be contradictory. For instance, words can state that the character is happy while the images show the character is upset. Verbal statements are more precise and concrete than images. Emotions are by definition nonverbal, and language can never convey an emotion effectively. This is where picturebooks offer a unique opportunity to engage with empathy and mind‐reading circumventing the inadequacy of language. Clever picturebooks make use of ambiguity created in the interaction between media when conveying a character’s emotional state.

In identifying an emotion from an image, we are likely to choose one of the basic emotions. For instance, we could, without the prompt from the title, read the cover image of Frog Is Sad as “The character is meditative” or “The character is bored,” but sad or distressed would be the most natural choice. Nuances of the emotion can be conveyed verbally with the range of sad, upset, melancholy, pensive, miserable, gloomy, unhappy, anxious, and so on. From images, we cannot exactly decide on the nuance. This allows a wider range of interpretation and subsequent response.

A verbal statement is more precise, but also restrictive. One might argue that verbal statements in picturebooks are redundant because images evoke emotional response more immediately and directly. Indeed, the aforementioned visual response method clearly shows that children have no difficulty in responding adequately to images unsupported by words. Most probably, verbal description of emotions and moods are used as a pedagogical device that ostensibly helps young children to articulate the emotion to recognize it later, but perhaps also as a step in language acquisition: from the basic “sad” toward a wider range of emotions.

Conversely, the interaction of word and image makes a connection between the vicarious emotional experience and its verbal description. When words are symmetrical with images, mind‐reading is restricted. However, to assume that a young child might not be able to read basic emotions without verbal support is dubious.

Unlike basic emotions, social or higher cognitive emotions, such as love, guilt, shame, pride, envy, and jealousy, are not innate, or at least considerably less innate than basic emotions, and they may be culturally dependent. The concept of social emotions emphasizes that they involve more than one individual and are thus subject to negotiations. Love is a social emotion requiring that two individuals’ ultimate goals, to be happy, become equally valuable for both. Social emotions are at least partly connected to basic emotions.

For instance, love can only lead to happiness if it is reciprocal. Unrequited love, on the contrary, leads to distress. Guilt comes from a contradiction between the basic emotion of joy and an action that disturbs the joy. Physical disgust corresponds to the social emotion of hatred. Envy and jealousy originate in frustration on observing somebody else’s happiness. This background from cognitive science is of overall importance for our understanding of how picturebooks communicate with readers and elicit their affective responses.

Although there are surprisingly many picturebooks featuring one single character, most of them involve at least two, which immediately brings in social emotions. John Burningham’s Granpa is a superb example of a picturebook focusing on social emotions, in particular love and guilt. Picturebook series such as Arnold Lobel’s Frog and Toad, Janosch’s Little Tiger and Little Bear, Mo Willems’s Elephant and Piggy, and James Marshall’s George and Martha, emphasizing interaction already in the titles, appeal to our tendency to anthropomorphize animals to introduce a wide range of emotions that might not be plausible with human characters.

Images play a significant role in representing social emotions and frequently carry the heaviest load, especially through body language and mutual position of characters on the page. Social emotions are not directly connected to external manifestations and thus more difficult to express visually. Although there are emoticons for all shades and degrees of joy, sadness, and anger, it is problematic to create a universal facial expression for envy or pride. If we need unequivocal visual signifiers for social emotions, we have to use symbols rather than icons—such as a stylized heart for love or skull for hatred—which are culturally dependent and generally need insider knowledge.

We can also infer, based on our real‐life experience or previous exposure to stories, that two characters in close embrace love each other and are happy together. Thus social emotions are more complex and composite; they can also be self‐contradictory and lead to positive as well as negative outcomes. Young children are initially solipsistic, but gradually become emotionally socialized, that is, trained to both express their emotions in a manner comprehensible for other people and understand other people’s emotions and empathize with them. It is frequently social emotions that feature in art and fiction; most of world literature is centered around at least one of them. Picturebooks are no exception, and examples are easy to find.

Finally, picturebooks involving several characters encourage young readers to engage in a more complex mind‐reading, or high‐order mind‐reading of the type: “A thinks that B thinks that A thinks….” In this process, readers are asked not only to understand what characters think and feel but also what they think and feel about each other’s thoughts and feelings. In real life, the main incentive of mind‐reading is predicting and anticipating other people’s actions and reactions through understanding their thoughts.

Picturebooks depict conflicts between characters based on misunderstanding, misinterpretation, and misdirection of emotions; they also depict characters developing empathy toward other characters. Such representations demand more sophisticated emotional response. In Where the Wild Things Are, for instance, a young reader may decide that the mother punishes Max because she does not love him. The readers’ empathy will thus align with Max’s anger, but they will not consider what the mother may be feeling, what Max thinks his mother is feeling, what the mother thinks Max thinks she is feeling, and so on.

Cognitive criticism claims that our brains can automatically process three to four embedded orders of mind‐reading. Without empirical evidence, I would guess that young readers do not go beyond two. This implies that they use immersive identification: They share the emotions of one character without trying to understand the thoughts and feelings of the counterpart. It is natural to empathize with the underdog, and in a child/parent conflict, it is natural for a young reader to empathize with the child.

Yet some picturebooks offer several choices for identification and therefore compel readers to engage in multiple mind‐reading. Anthony Browne’s Voices in the Park has been repeatedly used in empirical research with children to reveal their remarkable ability for embedded mind‐reading. But even considerably less sophisticated picturebooks, such as Frog and Toad, afford possibilities of more than one emotional perspective.

What then does cognitive criticism offer that more traditional thinking, such as reader‐response theory, does not? First, cognitive criticism confirms claims that were previously made without indisputable scientific evidence: reading fiction is not only beneficial, but indispenasble for our cognitive and emotional development. In plain words, reading makes us better human beings, which teachers certainly must seize upon (Kidd & Castano, 2013).

Second, cognitive criticism demonstrates that visual images are powerful means to invite readers to engage with texts and that picturebooks are perfect training fields for young people’s theory of mind and empathy. This potentially has wide implications for practitioners, because with this understanding, the importance of picturebooks reaches far beyond being reading matter for emergent readers. The common belief that picturebooks are intended for very young children and that school‐based literacy should primarily involve the mastery of verbal, written language and the supremacy of the verbal over the visual in formal Western education regrettably result in children losing their innate ability to engage with images—the ability to a high extent connected to affective responses.

Finally, then, cognitive criticism provides scholars and by extension teachers with concrete easy‐to‐use implements for studying young readers’ emotional engagement with picturebook texts. Seemingly simple questions such as What do characters feel?, How do we know what they feel?, and not least Why do we care about what they feel? probe into the most profound issues in the philosophy of art and its educational implications. They enhance young readers’ aesthetic appreciation of picturebooks.

Fictional narratives, including picturebooks, are about interpersonal communication, both within and outside the text. Representation of emotions in picturebooks enables communication when simple verbal description is insufficient. If, as cognitive criticism claims, we read fiction because we want to learn more about ourselves and about other people, picturebooks are an excellent first step toward emotional intelligence.