Embodiment and Image Design: From Mental Maps to Graphic Diagrams

This paper presents image making in communication design as diagrammatic modeling of mental maps. It demonstrates that any image is a diagram. Diagrams are examined as a mode of thinking that influences designs according to the logic of sensory perception.Thus, visual syntax is considered as the primary dimension of pictorial construction, which operates by the manipulation and placement of graphic signs.The stress is on the groundedness of semiosis in embodied experiences of reality, as reflected in cognitive processes, which are placed outside of the cultural realm.Therefore, a triadic model of a sign is discussed as a mental process, determined by species-specific cognitive and perceptual code.

Our connection with the environment is cognitive; therefore it is determined by the capacities of cognitive faculties, corporeality, and the interactive nature of the relation. During the evolutionary and developmental process of individuation we developed models for cognitive connection with the world. That is to say, mapping the sensory experience develops in accord to perceptual, intellectual, and operational schemas. They are “schematic structures that are constantly operating in our perception, bodily movement through space, and physical manipulation of objects” (Johnson 1987: 23). Those schemas, including image-schemas, are preconceptual plans or patterns of our expectations, anticipations, and conceptualizations of our interaction with environment. We can think of them as of dynamic models for organizing perception and experience, which allow us to make the world intelligible. This modeling entails relating things, events, and states.

Fig.1a Cognitive Mapping: interaction among an individual, the actual world, and the cognitive world.

© Elzbieta (Elka) Kazmierczak
2002
( Based on Ulric Neisser)

Cognitive theory enables us to view this modeling of mental schemas, as plans for conceptualization of past experience, which determine rational entailments described propositionally, and provide cognitive plans for interaction with the environment. We can think of a cognitive map as of a global “orienting schema” that consists of a number of other more specific schemas, such as perceptual schemas of objects. ”Actions are hierarchically embedded in more extensive plans and are motivated by anticipated consequences at various levels of schematic organization” (Neisser 1976: 113).

By Neisser’s view, the origin of cognitive and concept maps is in orientation (ibid.: 111): the concept map represents a gestalt of which only a fraction is per-ceived at any one time. For instance, knowing where one is entails a sense of what is around the corner. Cognitive mapping applies equally to concrete or spatial-temporal, and abstract realms. Thus, the correspondence between the mental pattern or a cognitive map and its representation defines different forms of expression, including different degrees of schematization and abstraction, such as “pictorial, hieroglyphic, alphabetic, schematic, and ultimately algebraic” (Merrell 1991: 263).

“A schema consists of a small number of parts and relations, by virtue of which it can structure indefinitely many perceptions, images, and events” (Johnson 1983: 29). That is to say, there are basic units of information/meaning, which are constituted by relations. Those schemas operate diagrammatically. They are diagrammatic in a sense defined by C. S. Peirce. Namely, as structural relations (Peirce CP 4.418). They are internal or mental diagrams general and abstract enough to allow us making connections among the richness and the variety of particular and concrete things, events, and states in the world. They are mental maps of our thinking. In order to communicate and to expand those models in some ways, we have developed external diagrams, as modes of representation such as mental, acoustic, graphic, or mixed.

The meaning-making strategies, or the ways we make sense of our experiences, are largely unconscious processes of mapping sensory experience onto the inner world of cognitive mapping. The metaphoric nature of that process refers to the “understanding and experiencing one kind of thing in terms of another” (Lakoff & Johnson 1980: 5). It describes the parallel (metaphoric) nature of the modeling process as such. For designers, the entailments of this mapping are intimately, if only intuitively known. For example, metaphor of navigation through the virtual space is taken from the experience of sailing and land exploration as in Internet Navigator, or Internet Explorer). Going through the pages of what actually are screen presentations derives from the experience with printed books. High or low impact of a design on a client is marked by experience of throwing objects. “Mood is an environmental state (as in ‘I’m feeling under the weather’)” (Danesi 1993: 121). The challenge is to turn intuitive knowing into the analytical and rational knowledge of visualization and meaning-making to produce predictable results and thus, to ensure the effectiveness of design.

Mental models can undertake different forms, such as iconicity, indexicality, and symbolicity. For instance, “perfumes are artificial icons of animal smells indicating sexual arousal or interest” (ibid.: 26). Perfumes symbolize elegance and femininity as well. Designs as external diagrams may acquire any modeling form. Therefore they vary on the scale of degrees of diagrammatic schematization. That schematization varies depending on the design purpose, medium, and the subject matter. It also varies in regard to the taxonomy of semiotic space in which they participate. For instance, geographic maps are indices.

The differentiation of the diagrammatic schematization can be organized into a scale from the most detailed realistic representation to the most synthetic abstract one (illustration 2a). For instance, a portrait of a person can be depicted as a photograph, as a painting, as a drawing, as a schematic, as a symbol, or finally, as an abstract mark (illustration 2b). The photographic portrait contains the most detailed depiction of the person’s appearance, while an abstract mark captures the essence of that person’s look or personality. Illustrations 2a and 2b show the progression of graphic schematization from experience to analytic abstractization of that experience.

Design develops diagrammatic representations of mental maps including perceptual schemas. In other words, the design process is the process of actualization of mental maps that takes place on two planes: on a mental plane (internal diagrams) of thought shaping and on the material plane (external diagrams) of its sensory counterpart. Design brings to existence mental diagrams of our conceptualizations of and about objects or events. These planes constitute the two modes of the diagrammatic modeling of thinking. They define the two aspects of the meaning-making process, which entails diagrammatic reasoning and its representation. We can think of a reasoning itself, which involves making conceptual relations by spatial means, as of “‘mapping’ in its hypothetical sense” (Spinks 1991: 446). Thus, the design is the mapping in its actual sense. In other words, it is a process of representing conceptual relations by spatial means in a graphic or other medium.

Design relies on selection and schematization of sensory characteristics to enable receivers to conceive of something quite beyond what is actually seen. Thus, the design is a diagrammatic guide for the imaginal construction of a meaning: concept or entity. Design is mapping in the diagrammatic sense.

Designs are shortcuts through and to meaning. They rely on mutual reinforcement of pictorial and textual components, which takes advantage of cognitive differentiation between processing of different types of sensory information. They show what is meant and thus, take advantage of the efficiency with which humans process visual information. Linguistic messages are processed more slowly as they require sole intellectual processing which takes longer. Graphic diagrams, represent the way humans think, therefore they are comprehended quickly. Designs do not represent objects as they are in and for themselves. They are schematics. Designers strategically bring into designs only those aspects of the object that are essential to the design objective. The design process is driven by dichotomy of chosen versus rejected characteristics of an object. Designs do not rely on lengthy descriptions or specifications, as do textual communications. Instead, they provide selections of sensory cues necessary for the immediate grasping of an adequate interpretation. Graphic diagrams provide a geographical overview of relationships among corresponding relevant concepts. They are sensory schemas for comprehension of conceptual relations.

The development of the graphic diagram requires strategic thinking and planning that requires intellectual discipline and reductive reasoning. Viewing design process as an extension or result of the diagrammatic reasoning allows us to emphasize the importance of the rational foundations of that process.

In illustration 4, the viewer navigates through the arrangement being guided by its visual hierarchy. The viewer’s reading of the diagram is always non-linear, associative and abductive, which makes it different from the linguistic comprehension. It is the dynamic relation of mutual determination between the logic of visual cognition and the linguistic discourse that determines reading of diagrams.

Pictorial elements, especially those with a high degree of generalization, require linguistic support to specify referents. In this sense, diagrams are conceived as open to different readings, but those readings are limited by the logical possibilities afforded by visual syntax. The linguistic elements specify referents. The pictorial/graphic elements specify relations. The same applies for pictorial elements with a low degree of generalization, and a low degree of schematization, such as realistic images or photographs. For instance, different titles specify different referents, and thus, different readings, of the same picture. Mona Lisa will “read” differently in an art history book, in a costume design book, or in a teenage girls magazine as an example of a poor make up.

There are two aspects of graphic diagrams that are important to visualization of conceptual structures. First, logic of visual syntax (formal relations) guides the sequence of the reading of a diagram. Secondly, diagrams are gestalts, and as such, fall under the laws of visual perception. In other words, besides being objects for communication, diagrams are objects for cognition. As spatial models, they create sensorial configurations, which operate semiotically to communicate conceptual relations. Therefore graphic diagrams operate through visual cognition.

Gestalt psychology describes laws of visual grouping, explaining that the whole of the visual arrangement determines perception and comprehension of its components. Perceptual processes as operating according to perceptual schemas orient us to physical reality and function consistently across cultures.

to top

In this illustration, one square appears behind the other. Once we grasp a configuration, that comprehension encompasses and specifies the structural relations inside it. Thus, we perceive gestalts top-down or general to specific. Graphic diagrams are holistic and synoptic, like any other sensory arrangement, in a sense that they are perceived always as wholes or gestalts. A receiver looks for a familiar pattern to decide about the relations within it. Namely, the interpreter looks for the meaning of the design elements upon comprehending the whole first. Gestalt psychology explains why diagrams enable top-down comprehension of unfamiliar content and support remembering of complex relations (Holley & Dansereau 1984: 14). When applied as teaching or learning aids, diagrams are excellent tools facilitating meaningful learning and remembering of the unknown material. Graphic diagrams provide spatial models of conceptual structures and thus facilitate understanding and remembering of otherwise difficult to grasp concepts.

Graphic diagrams accomplish so much, because they materialize the spatial and temporal ways we think. That means, that at the heart of diagrammatic models are visual relations as ways of thinking. In graphic diagrams, “the spatial relations between their tokens share logical properties with relations between denoted objects” (Gurr 2002: 128). That is to say, there is a direct mapping between concep-tual relations and relations shown in graphic diagrams. As a result, in diagrams, “certain inferences are somehow more immediate, or even are automatic”, and “conclusions appear ‘for free’, as compared with textual systems where a logical inference must be made to produce the conclusion” (ibid.: 131). In the illustration 7, we see the most rudimental diagrammatic notations of basic conceptual rela-tions: connection, mutual causation, clustering, overlap, and inclusion. Once again, they show what they mean. They operate not by resemblance to appearances. They are visual representations of diagrammatic reasoning operating according to the rules of visual logic guided by perceptual schemas.

Thus, in spite of the theoretically unlimited possibilities of assigning meaning to abstract shapes, an arrow lends itself for the use in those situations in which the logic of the syntax of an arrow is applicable. In short, an arrow always implies a specific direction, motion, and thus, possibility for transition. It is important for a communication designer to be acutely aware of the semantic entailments of graphic shaping. Namely, she/he must keep in mind that the changes in the sensory organization of gestalts create different patterns for recognition. Different sensory configurations are recognized as different signs (diagrams), which may involve different interpretations, or inferences. For instance, change of a position of the same triangle, always changes its indexical function. Namely, it points to-ward a different direction every time it is turned.

The graphic page itself is diagrammatic. The spatial coordinates of height and width define it. This plane maps our position in the world, which is mapped in relation to our orientation within four directions. Geographic directions – north south, east, west, or corporeal directions – forward, backward, right, or left are mapped onto the plane of a graphic page as up, down, right and left. In this context it is easier to see the diagrammatic nature of text layouts, which are diagrams as well.

The Meridian poster is a concept diagram for connecting fragments of infor-mation on a topic. It uses the code of symbolic representation by integrating text as if it were an image. The complexity of the architecture causes the viewer to consider syntactic relationships as guides to semantic associations of texts. The viewer is guided to make conclusions about conceptual relations using cues from visual syntax or composition. That is because sensory proximity and perceptual dynamics determine what we link together. Whatever we link sensorially, we can link conceptually. Height and width correspond to the surface of the earth. The visual syntax leads the viewer to combine texts, and in doing so, to build a semantic field or topic. The rules for reading texts are not explicitly stated. Instead, a complex logical layering of information according to proximity and directionality indicates them.

to top

This map is an indexical diagram. This diagram is indexical by virtue of having a one-to-one mapping with referents: road crossings. The value of this diagram is not in its resemblance to the appearance of the urban landscape, but as a guide to the schematic patterns of connections among streets and routes that can be inferred.

It is a frequently used practice in diagrammatic modeling to bring together mutually excluding viewpoints, such as, for instance, in this also indexical diagram of polar projections (illustration 11). It mixes different projections, which show different points of view. These polar projections are developed for different purposes. Both are accurate, but from different frames of reference. Thus, what looks distant in one projection looks adjacent in another. It is a concise synopsis, which provides two different points of view simultaneously.

Illustrations are diagrams in disguise. They may not look like diagrams, but they work diagrammatically. In Addicted we see an illustrative diagram. It uses the convention of realistic representation of mechanical projection of photography and assemblage. Here the receiver is confronted not with a portrait of a man, but with the diagrammatic representation of the personality shattered by the addiction. The pulling of the contradictory forces inside the head is visualized literally by the strings hooked to the head. There is also a blend of mutually excluding viewpoints. The image simultaneously shows the views of the outside and the inside of the head of an addict. Once again, illustrative diagrams do not arrive portraying anything we see in reality. They use a “lifelike” way to communicate diagrammatically.

Another illustrative diagram, a cultural poster, shows a relation that challenges our sense of balance. In a synoptic overview, the design forms an up-side-down pyramid. We see the lion’s expression, we see its teeth, wee see its precarious perch. The instability is achieved not only with scale and proportion, but also with indication of movement by arched spikes. The linguistic component “cyrk” (circus) provides the referent for the lion, thus, giving the receiver the cues for the referential meaning of the poster.

This medical illustration may not look like a diagram, but it works diagrammatically. This is not a portrait of a bacteria. It is a depiction of structural relations and schemas. This is a diagram in which certain aspects are emphasized to make clear the fitting of the eColi bacteria's receptors to the lining of the urinary tract (cued in red).

Summary

to top

1. Designs vary on the scale of degrees of diagrammatic schematization. That schematization varies depending on the design purpose, the subject matter, and medium.

2. Designs vary in regard to the taxonomy of semiotic space in which they participate. Thus, the correspondence between mental diagrams and their graphic representations motivates different forms of diagrammatic expres-sion, such as indexical, illustrative, flow chart, and concept.

3. In a reductive, condensed, and synoptic way, they show only those fea-tures and aspects of objects or events, which guide the receiver’s involvement.

4. Graphic diagrams do the following:
(1) depict relations & schemas, not appearances,
(2) reveal underlying conceptual relations and behaviors,
(3) provide new con-clusions or statements that convey points of view, and
(4) represent con-ceptual relations by spatial and temporal means.

Danesi, Marcel. 1993. Messages and meanings: an introduction to semiotics. Toronto: Canadian Scholars’ Press Inc.

Gurr, Corin. 2002. "Combining semantic and cognitive accounts of diagrams", in: Anderson, Michael, Bernd Meyer & Patrick Olivier (eds.). Diagrammatic representation and reasoning. London: Springer-Verlag London Limited.

Holley, Charles & Donald Dansereau (eds.). 1984. Spatial learning strategies: techniques, applications and related issues. Academic Press Inc. Harcourt Brace Jovanovich Publishers.

Johnson, Mark. 1987. The body in the mind: the bodily basis of meaning, imagination, and reason. Chicago, Ill: The University of Chicago Press.

Lakoff, George & Mark Johnson. 1980. Metaphors we live by. Chicago, Ill: University of Chicago Press.

Merrell, Floyd. 1991. "Model, world, semiotic reality", in: Merrell, Floyd & Myrdene Anderson (eds.). On semiotic modeling. Berlin & New York: Mouton Gruyter.

Neisser, Ulric. 1976. Cognition and reality: principles and implications of cognitive psychology. San Francisco, CA: W. H. Freeman and Co.

Peirce, Charles Sanders. 1931-35. Collected papers. Hartshorne, Charles & Paul Weiss (eds.). Cambridge, MA: Harvard University Press, Vol. 4, 418.

Spinks, C. W. 1991. "Diagrammatic thinking and the portraiture of thought", in: Merrell, Floyd & Myrdene Anderson (eds.). On semiotic modeling. Ber-lin & New York: Mouton Gruyter.