Plass and Kaplan state that the goals of their chapter are to
(1) review basic concepts related to emotion and learning, (2) summarize research on emotional design in digital media for learning, (3) present a theoretical framework of learning from digital media that emphasizes the need to consider emotional design factors in addition to cognitive design factors when designing multimedia learning materials, and (4) develop a research agenda for the study of emotional design for multimedia learning. (p. 1)
A substantial portion of Plass and Kaplan’s discussion in pursuit of these goals involves their argument that emotional or affective considerations should complement cognitive considerations when designing genuinely effective multimedia learning material. Such an approach, they note, that diverges from more traditional multimedia design for learning, which often ignored or minimized the affective in favor of the cognitive. Cognition and emotion, they note, are “inherently interconnected,” and “[t]his interconnectedness is an essential aspect of the complexity of human consciousness” (p. 1).
The interconnected relationship between emotion and cognition is a dynamic one, wherein “dynamic cognition-emotion interactions… emerge and operate in ways that are highly contextualized (and hence sensitive to contextual factors)” (p. 22). These interactions, Plass and Kaplan conclude, “serve as motivating forces that guide human adaptation and learning in specific contexts” (p. 22).
In their 2008 article, Knez and Neidenthal state that their study on lighting in digital game worlds “was designed to investigate the impact of warm (reddish) and cool (bluish) simulated illumination in digital game worlds on game users’ affect and play performance” (p. 129). They sought to compare these impacts with similar impacts (on affect and performance) observed in “real-world” environments. psychological effects of such lighting in “real-world” environments. What Knez and Deidenthal found was that users playing in digital worlds
performed best and fastest in a game world lit with a warm (reddish) as compared to a cool (bluish) lighting. The former color of lighting also induced the highest level of pleasantness in game users. A regression analysis indicated tentatively that it was the level of pleasantness induced by the warm lighting that enhanced the players’ better performance in that digital game world. (p. 129)
One of the things I found interesting about this article was a point that initially felt counter-intuitive to me, but quickly made a great deal of sense and left me wanting to investigate the subject. As Knez and Neidenthal note,
high-skilled players were…much more precise in their digital game taste than were the medium- and low-skilled players were. They played only the FPS (first person shooter) and RPG (role playing game) types of games (80% vs. 20%), while the medium- and low-skilled players played FPS, RPG, RTS (real-time strategy), action, sports, adventure, consol[e], music, hearts, MMORPG, Sim, and puzzle types of games. (p. 133)
What (point or process) causes a gamer’s shift from imprecise to precise tastes is one that I suspect the commercial games industry has investigated thoroughly. But has the educational games industry done the same? And why exactly are medium- and lower-skilled players more varied in their tastes? Are the reasons few or many?
In this chapter Picard frames her discussion with what she—and most others—see as the two primary aspects of human emotion—the physical an the cognitive. Within—and across (as the distinctions between “physical” and “cognitive” are far from clear-cut)—these two aspects Picard explores particular features of human emotion (or “emotion theory”) she feels are important to consider in order to create genuinely affective computing, to give computers “the ability to recognize, express, and ‘have’ emotions” (p. 44). These features are “social display rules, universal vs. person-specific responses, primary vs. secondary emotions, the role of emotions in creativity and general memory processes, the existence of multiple paths for emotion expression in humans, and emotion inducement” (p. 44).
Picard also spends some time highlighting ways in which “emotion can be expressed through sentic modulation—including facial expression, vocal intonation, gesture, posture, and other bodily changes” (p. 44). “Sentic modulation,” in turn, are expressions “such as voice inflection, facial expression, and posture,” are “the physical means by which an emotional state is typically expressed, and is the primary means of communicating human emotion” (p. 25).
One of the things I found particularly interesting about this chapter is Picard’s consideration of computers’ “recognition problem”—wherein they have (or at least had, at the time of the book’s publishing) difficulty recognizing, or focusing-in on and interpreting, human speech acts and meaning under certain circumstances. She argues for a
“universal” recognizer… [that] would first ask “Which category is this person most similar to?” In speech, this might be likened to asking “who sounds like this—both accent-wise and voice-quality wise?” Subsequently, a recognizer can be used that was trained on the prototype person for that category. A benefit of this approach is that it is also likely to reveal categories of affective expression that theorists have not yet identified. (p. 34)
I’d never thought of such a problem before, and it leaves me wondering what kind of recognition systems currently exist, and how much they do or don’t resemble such a kind as Picard describes.
In the Introduction to Affective Computing (1997), Picard overviews and sets up her argument for the book, stating that it “proposes that we give computers the ability to recognize, express, and in some cases, ‘have’ emotions” (p. 1). She proposes this because, she contends (after raising and critiquing various points that might be made against such a proposal), “[a]fter nearly a half century of research … computer scientists have not succeeded in constructing a machine that can reason intelligently about difficult problems or that can interact intelligently with people” (p. 1). Picard further sketches the outlines of the book as “lay[ing] a foundation and construct[ing] a framework for what I call ‘affective computing,’ computing that relates to, arises from, or deliberately influences emotions” (p. 3).
Much of the rest of the Introduction is devoted to Picard making the case for (simply put) emotional intelligence—both as a vital element in human cognitive functioning (including decision-making, a subject she spends some time on here) and as a form of intelligence we should strive to create for and in computers. “[C]omputers,” she argues, “if they are to be truly effective at decision making, will have to have emotions or emotion-like mechanisms working in concert with their rule-based systems” (p. 12).
One of the things that struck me about Picard’s argument in the Introduction is this phrase, “emotion-like mechanisms.” At this—early—point in her book, I can’t decide whether her offering it is a compromised position (from computers having “real” emotions or emotional capabilities) or not, whether it seems to come somewhat too soon or somewhat too late in her argument, even though it’s only a dozen pages into the book.