Synesthesia: Beyond the Five Senses
by Oyang Teng
Gottfried Leibniz once wrote that our sense perceptions are occult qualities, whose familiarity does nothing to render their essential nature more intelligible. After all, each person's sense experience is ultimately subjective. And despite the fact that language establishes a correspondence between the sensations of each and all, could anyone, for example, precisely define the color red?
Nevertheless, as Leibniz conceded, the study of human perception does yield important truths about that aspect of the physical world represented by our physiology, and the way the mind deploys such physiological functions to construct knowledge of the universe. That is, as messengers, we receive through our sensorium, "as through a glass darkly," the distorted shadows of the external world, shadows whose mutual interplay—and mutual contradictions—prompt our mind to discover order in the world.
Long before brain-imaging technology showed that even basic perceptual acts involve many different areas of the brain, common observation (and common sense) showed that there is no strict autonomy of any of the senses; rather, they each exist as interconnected aspects in a continuum of perception. Consider the following, from psychologist Erich von Hornbostel's 1927 paper, "The Unity of the Senses":
Here is a tone, here are a number of different grey papers from black to white; choose the one which is as bright as the tone. This one? (Indignantly) "Too dark!" This one? "Too bright!" That one? "Still too bright!" And so on. It can be done quite easily and with great precision; and everyone, except the colour-blind, can find a grey to match the tone. Furthermore, anyone can find on the piano that tone which sounds as bright as lilac smells. (Generally he thinks the task nonsense at first, but, if he can be persuaded to deal with such nonsense at all, it goes very well.)
So there is a "sensuous" which is not limited to one single sense. Indeed, looking more closely, the apparent exception becomes the rule, and one must search in order to find the private property of any one sense.[1]
The unity of the senses playfully described by Hornbostel, is perhaps nowhere more dramatically expressed than in the recently studied condition known as synesthesia, which, though acutely experienced by relatively few people, demonstrates more universally that the degrees of freedom of even the five traditional senses of human perception extend far beyond conventional accounting.
Synesthesia and the Mind's Eye
People with synesthesia experience a blending of the senses, such as sight and hearing, or a blending of characteristics within a sense modality, such as associating colors with written letters. More precisely, synesthesia occurs when "a triggering stimulus evokes the automatic, involuntary, affect-laden, and conscious perception of a physical or conceptual property that differs from that of the trigger."[2] In other words, it can involve not only the union of two different sense modalities (and, in rare cases, more than two), but also different dimensions of perception, such as spatial extension, as well as affective characteristics like personality or gender.
Some form of synesthesia is currently estimated to occur in roughly 4% of the population, the most common form being the experience of color for days of the week, followed by colored graphemes (a unit form in writing, usually letters or numbers), in which the color appears whether the grapheme is read, heard, or merely thought of. In this case, it is the concept, and not the literal shape, of the grapheme that is important, indicating that synesthesia involves both lower-level perceptual processes, such as the recognition of color, and higher-level processes more directly influenced by cultural development and language. For example, one experiment used a distorted shape that in one context could be interpreted as an "A," and in another, an "H," and the respective synesthetic association was evoked in each case.
Synesthesia is far more common in children than adults. It is thought to occur universally in infants, reflecting a brain that is still in the process of differentiating the combined sensory experiences that characterize the infant's purblind state.
Other common synesthesias include number forms, a specific type of spatial sequence synesthesia, which is any that involves the combination of color and spatial configuration with concepts involving sequence; tasted words, triggered by either spoken or written words; colored hearing, involving the elicitation of color, shape, and movement by sound, whether environmental or musical; and the personification of letters and numbers. A typical example of the latter: For one woman, the letter V is "yellow beige but subdued (more beige than A; deeper than S; more yellow than L or K); female; very feminine, unflauntingly sexy, sophisticated."[3]
Beyond these more common types, just about every other sensory combination has been documented, including tasting shapes, feeling musical notes, and hearing temperature. With the exception of cases of sensory overload, most synesthetes report that the experience is pleasant, and in general, it is found to help with memory, because of the multiple perceptual associations bound up with certain events or experiences. Musical memory is no exception: In a skill that many would envy, some musical synesthetes can automatically check their pitch or key based on its characteristic color or taste.
Lying somewhere below the threshold of the percepton of an actual external stimulus, but above that of a merely imagined effect, synesthetic perception provides a striking insight into the operation of the "mind's eye" (and ear, nose, etc.). That is, the experience is real, immediate, and vivid, but at the same time, synesthetes can easily distinguish between the stimulus and the effect, so that an evoked color, for example, doesn't obscure one's visual field, and isn't mistaken for some kind of actual floating, colored object in space.
It is also worth considering to what degree synesthetic perceptions are more than the sum of their "parts," constituting perceptual categories for which we, as yet, have no names, as the analogous case of the relationship between a bird's vision and its magnetic sense demonstrates: Is it sufficient to say that it simply "sees" the magnetic field?[4] Other cases call into question the very meaning of existing definitions, as in the case of auditory effects from aurorae or meteors whose cause cannot be attributed to those associated with conventional hearing.[5]
The types of synesthetic sensory experience capable of spatial extension, also have the interesting property of expressing certain ordered configurations, rather than completely random and arbitrary associations. These "form constants" were first systematically catalogued by the German Gestalt psychologist Heinrich Klüver, and include such basic geometric forms as tunnels and cones, central radiations, gratings and honeycombs, and spirals, in various degrees of motion or pulsation, and can also occur in non-synesthetes under a variety of conditions. Again, such forms can apply to all experiences of spatial extension, not just visual experience. For example, someone with tactile synesthesia might feel what seem to be regularly ordered geometric shapes. This suggests that organized perceptual wholes, or gestalts, are not confined to specific sense modalities, and might offer clues as to the formation of gestalts which transcend perception per se, gestalts which are involved in such creative processes as artistic composition.
What is the underlying neural basis for synesthesia? Brain-imaging studies have shown increased "cross talk" between areas of the brain associated with the sensory functions implicated in different kinds of synesthesia, but it is not known for certain what causes the increased cross talk among different areas of the brain in synesthetes. It is likely that synesthesia simply unmasks commonly existing neural pathways, which research continues to suggest are far more interconnected throughout the entire brain than previously thought. There are a number of circumstances under which synesthesia can be temporarily acquired (such as during the period between waking and sleeping, when sudden noises can trigger perception of a burst of color) or consciously induced (as by meditation or drug use). Sensory deprivation, as in experiments with subjects blindfolded for several days, also causes a re-appropriation of the visual cortex to hearing and touch, a neural reorganization that becomes permanent in those who go blind.
Regardless of the particular brain processes involved, there is plenty of evidence for common synesthetic—or, at least, intersensory—perceptual patterns shared by all people, of which only a few examples follow.
The Common Sense
In the 1920s, Wolfgang Köhler, one of the founders of Gestalt psychology, demonstrated the tight connection between sight and hearing, in a famous experiment in which subjects were asked to identify which of two figures—one angular, the other rounded—was named "Takete," and which was named "Maluma" (Figure 1). An overwhelming number of test subjects associated the angular figure with Takete, the rounded one with Maluma, showing common associations exist linking the sound of certain figures and their corresponding visual representation.
Another experiment on the mutual influence of sight and sound shows that the way a person's lips are perceived to form a sound can override the auditory stimulus itself, a phenomenon called the McGurk effect. For example, if the sound "ba" is heard corresponding to the image of a person mouthing "fa," the sound will be heard as "fa." More familiar cases also illustrate auditory-visual association, as happens every time we watch a movie, and believe the sounds emanating from the audio speakers to come from the action on screen.
Perhaps not surprisingly, spatial association with numbers is also common. In what has become known as the snarc effect (spatial numerical association of response codes), subjects are shown two numbers and asked to push a button corresponding to the larger. There is a direct relationship between the rate of response, and whether or not the larger number corresponds to the direction of larger numbers on an imagined, spatially extended number line. In synesthetes, these number forms are experienced as explicit projections in the space around the person's body.
Studies have long shown that pitch and color are also intrinsically linked. Synesthetes and non-synesthetes alike universally associate higher, louder tones with lighter colors, and lower, quieter tones with darker colors (Figure 2). Taste and smell have likewise long been associated, and simply holding one's nose while eating is enough to demonstrate their interdependence.
A recently published study showed that three-dimensional shapes, normally an inherent property of vision and touch, can be represented by artificially coded sounds. This "hearing of shapes" is treated by the authors of the study as a further demonstration of the natural "metamodal representation" of most perceptual processes.[6]
Of course, one need look no further than our use of language to see the deeply imbedded integration among the senses: whether describing someone's loud shirt or dark mood, the soft tones of a musical piece, or the "wonderful rhythmical flow of lines and curves" in sculpture, "more subtly felt than seen," as Helen Keller described it.
Keller's case is particularly interesting. Although she sensed the world almost entirely by touch, her use of language contains such vivid and sensuous imagery that one wonders whether it reflects a kind of synesthesia, or simply her amazing ability to absorb all that she read. In any case, there is no way to be certain what resemblance, if any, the "colors" in her mind's eye, or the "sounds" in her mind's ear, would have had to the corresponding senses of those with sight and hearing.[7]
There is also a reciprocal influence of language on perception. In one 2007 study, researchers showed that blue color discrimination was different for Russian and English speakers, based on the fact that Russian makes a categorical distinction between lighter shades (goluboy) and darker shades (siniy) of blue.[8] Like synesthesia itself, in which triggering stimuli are often products of learning and language, this suggests a dynamic interplay between culture and perception, in which our senses, rather than being "hard-wired," are instead somewhat conditional, subordinated to the continuing evolution of our cognitive powers. Other phenomena, such as sensory substitution—for example, the use of a device that produces tactile sensations on the tongue to simulate vision, or, alternately, the development of echolocation abilities in the blind—further underscore this point.[9]
Some have been prompted to conclude that synesthetic associations at the foundation of perception may have been necessary for the development of language itself:
Marks concludes that perceptual experiences of meaning are multidimensional and that verbal (semantic) knowledge taps earlier perceptual knowledge. This conclusion is echoed by Sean Day who notes that colored sounds are the most common expression of perceptual synesthesia, whereas metaphoric elaborations of tactile sound are most common in (English) literary synesthesia. It appears likely that human thought itself is largely metaphoric. Hearing is the sense most frequently expanded by both perceptual synesthesia and synesthetic metaphors. Sean Day also concludes that synesthetically seeing sounds, which antedates language, has probably influenced language development.[10]
It is important to note, however, that metaphor is not a mere epiphenomenon of cross-sensory perception—it is, rather, given in the very structure of the universe as we know it. The idea that the universe can be known literally, is an artifact of the naïve presumption that our senses should somehow convey, even if only ideally, a more or less direct picture of reality. But true knowledge cannot be literal; it is only through paradox, through the principle of contradiction, that universals are known and communicated: what Nicholas of Cusa precisely termed "learned ignorance." This ontological principle of mind demands a corresponding form of organization of our neurological and perceptual apparatus, and it is this top-down requirement that makes possible all of the phenomena so far described.
From this standpoint, language is necessarily metaphoric. Despite the objections of positivists and others similarly wedded to sense certainty, the ambiguities that words inherently embody, represent, not a limitation, but rather a reflection of the interplay of the infinitely subtle "shadows" that form the backdrop to human thinking. Since no object of sense perception is self-defined, no language—mathematical or otherwise—can ever reach a state of completion.
Neither, it seems, can our own sensorium.
Additional References
Noam Sagiv & Jamie Ward, "Crossmodal interactions: lessons from synesthesia," (Progress in Brain Research, 2006).
"Red Mondays and Gemstone Jalapenos," Research Channel documentary, 2009.
"Derek Tastes of Earwax," BBC Documentary, 2004.
[1] Erich von Hornbostel, "The Unity of the Senses," 1927.
[2] David Eagleman and Richard Cytowic, Wednesday is Indigo Blue: Rediscovering the Brain of Synesthesia (Cambridge, Mass.: The MIT Press, 2009).
[3] Ibid.
[4] See Ben Deniston, "Magnetoreception," in this issue.
[5] See Sky Shields, "Unheard Melodies: Electric and Magnetic Sense in Humans," in this issue.
[6] Jung-Kyong Kim & Robert J. Zatorre, "Can you hear shapes you touch?," (Experimental Brain Research, 2010).
[7] See Meghan Rouillard, "Helen Keller: Mind Over Instrumentation," in this issue.
[8] Jonathan Winawer et. al., "Russian blues reveal effects of language on color discrimination" (PNAS, 2007).
[9] Indeed, what might all of this imply about the brain's capacity to be "tuned" to different modes of perception, such as through the still-poorly understood effects of different types of electromagnetic radiation?
[10] Eagleman and Cytowic, op. cit.
