Skip to navigation – Site map
I. Articles on synaesthesia

More than a condition: an examination of synaesthesia as a key cognitive factor in the processing of reality and in its literary and pictorial renditions

Séverine Letalleur-Sommer

Full text

What seems to influence language at its most profound level,
is the Lebenswelt, the experience of the body
in its relationship to the environment and to others. […]
Meaning is thus firstly anchored in anticipation, qualitative,
often synesthesic feelings, and not in a directional grasping of “objects”.
(my emphasis; Cadiot 41)

1The issue of synaesthesia, whether it be addressed as a benign anomaly or an artistic device is a highly composite and heterogeneous matter. It is more complex than the mere amalgamation of sensory modalities ordinarily considered as distinct. Part of the difficulty in elucidating it derives from the fact that the correlated modalities may (or may not) vary in nature, in scope and even in essence - cultural symbols being at times related to seemingly more trivial phenomena like digits associated with texture for instance.

2Yet, because it features processes that combine the concrete and the abstract, percepts and concepts alike but also affects, in the case of personifications for instance, synaesthesia is of particular relevance in the field of linguistics and semiotics. It is indeed tempting to draw a parallel between those irrepressible cross-modal reactions, that bear no obvious link with their inducers, and the way arbitrary linguistic signs interact with their ideational component. The same processes may lie at the core of both phenomena, and synaesthesia could be an atypical remnant of otherwise common and widespread meaningful mental associations. Similarly, it could be of interest when considering in what way perception is altered by symbolic, cognitive and possibly emotional content and vice versa. This is all the more relevant as if “vivid” or real synaesthesia (to use the words of psychologist Lawrence E. Marks [2011, 51]) only affects a relatively small percentage of the population, in its “relative” form, it is pervasive and not that remarkable.

3In what follows, my aim is to explore the hypothesis according to which synaesthesia might be a way of perceptually securing data that is too elusive or abstract to be easily recorded in the memory and therefore understood, as if the purpose of it all was to “flesh out” otherwise transient and impalpable semantic or proto-semantic content. In that respect, it would amount to a cognitive process made to encode, store but also retrieve and express meaningful experiences better; hence its omnipresence in art and literature where expressiveness plays a crucial role.

Scientific data

4Amongst the various depictions of synaesthesia encountered throughout this research, one of the most accurate and concise reads as follows: “People with synaesthesia have anomalous perceptual experiences that are triggered by activity in another sensory modality (e.g. sounds triggering colours as well as auditory experiences) or by other cognitive activity (e.g. numbers triggering colours).” (Ward & alii 128) In other words, synaesthesia is widely regarded as an unusual psychophysical association where a specific stimulus, often referred to as an “inducer”, entails a superfluous sensory reaction, a “concurrent” or “inductant” (Marks 2011, 66), on top of the perception normally elicited. Both components can differ in nature, when a sound occasions a visual representation for instance, or not; thus, in the case of grapheme-colour association, both inducer and concurrent – here also referred to as photism - are of the visual type. Moreover, the bond between inducer and concurrent may occur via physical presentations of the inducer or via more internal, mental, endogenous representations, which adds to the complexity of the issue:

A minority of synaesthetes (11 out of 100 in our sample) have described their colors as being “out there on the page,” as though a transparency bearing a colored number was placed on top of the written digit. We call these synaesthetes projectors […] In contrast, the majority of synaesthetes we have interviewed have described their photisms as being “in my mind’s eye” or “in my head.” Their photisms consist of internally perceived colors, rather than externally perceived colors […]. We call these synaesthetes associators, because their descriptions of their experiences reflect a strong association between graphemes and specific colors. (Dixon & alii 335-336)

5The inducer/concurrent terminology is sometimes called into question though, for it is regarded as too restrictive to properly account for the underlying processes (Rosenthal 2011a, 10). And it is true that inducers and concurrents not only greatly vary from one synaesthete to another, but also hardly ever amount to the sole stimulus/response type of affiliation, for the abnormal bond may range from clusters of homomodal or heteromodal sensory modalities (visual/visual, sound/visual) to culturally-bound clusters combining time-units and colours, or more elaborate qualia combinations involving pain, orgasm and kinetics for instance (see American psychologist Day’s online table); all the more so as many synaesthetes testify to the inherent quality of both terms thus contradicting the idea that they would supplant one another.

6When examining Day’s 63 potential associations, one can see that, amongst synaesthetes, some associations are recurrent whereas others are rather scarce. Grapheme-colour synaesthesia is thus commonplace but there exist more occasional and intriguing types, for example when a specific tonality causes a specific taste or flavor in one’s mouth: “hearing a major-sixth tone interval produces the taste of low-fat cream, and hearing F-sharp produces a purple colour experience” (Simner 23). Besides, the process is hardly ever bidirectional which means that if an inducer triggers a specific concurrent, the latter will barely ever elicit the original inducer in return (Mächler 5). We shall come back to this later as it requires closer attention.

7Day divides the various phenomena into two sub-categories, one referred to as “synaesthesia proper,” where “stimuli to a sensory input will also trigger sensations in one or more other sensory modes”, and the other referred to as “cognitive synaesthesia” which encompasses cases where abstract sets of symbols, that are often hierarchically or at least orderly organized (such as numbers, letters or time-units) are systematically associated to more mundane sensory modalities such as colours, tastes, touch or flavors; the same distinction is put forward by two other American psychologists, Grossenbacher and Lovelace, but termed differently: “In synesthetic perception, concurrents are induced by perceiving particular sensory stimuli. In synesthetic conception, concurrents are induced by thinking about particular concepts.” (Grossenbacher & Lovelace 36).

8What is telling when considering the table mentioned above, is that the visual dimension is overwhelmingly represented, supporting the prevalent idea according to which “sight is the strongest of all the senses” (Yates 19) – at least in Western societies. Indeed, if we consider Day’s statistics, amongst the approximate 4% of the population affected by synaesthesia, 61.67% associate graphemes and sounds, 15.09% general sounds to vision and 18.57% musical sounds to vision; whereas other kinds of synaesthesia are comparatively far less frequent (all below 5% and most below 1%). Those figures have to be handled with caution though for most synaesthetes are unaware of their condition as something of a rarity, and may remain ignorant of it all their lives so that, lest they should be detected, most of them will stay unidentified and uninformed.

9Another interesting statistical cue is that synaesthesia proves far more prevalent in females than in males with a sex ratio of 6:1 (Baron-Cohen & alii 1996, 1073; Barnett 877). Also, if the condition seems to run in the family, it develops differently in each individual, including among those who are relatives: “[…] various types of synaesthesia are fundamentally related at the genetic level, but […] the explicit associations and the individual differences between synaesthetes are influenced by other factors” (Barnett 887); which means that even if people who are prone to synaesthesia have inherited their gift or condition from close-related ancestors, it will prove highly idiosyncratic, and significantly vary according to each individual.

10Most studies indicate that the phenomenon is spontaneous, reliable and irrepressible, while remaining exceptionally consistent over time. Such stability is actually one of the main criteria scientists will have recourse to in order to ascertain the genuineness of what is experienced by synaesthestes. For the latter will systematically associate this or that colour or hue, to this or that letter, in the case of grapheme-colour synaesthesia for instance, over and over again, even months or years after they were first spotted and interviewed.

11On the whole, instead of underscoring an odd combination between two sensory modalities, the data collected and briefly summarized above, points to the fact that synaesthesia often associates symbolic forms with several sensations in the case of grapheme-colour synaesthesia, or time units and colour synaesthesia for instance; that the phenomenon lies beyond the scope of strict sensory modalities with kinetics and spatialization coming into play as well as socio-cultural, cognitive aspects; and that if it is highly specific, varying from one individual to another, it is more likely to be found in women than in men.

Hypotheses from the classic literature

12Although the data collected up until now still does not succeed in giving clear explanations accounting for the condition, numerous thought-provoking hypotheses have been put forward, some of which I will list in the following paragraphs. As hinted at earlier, all the hypotheses suggested seem to revolve around the long-standing nature-nurture debate with assumptions ranging from genetic factors to behavioral and learned associations.

13Recent studies have detected one or several genes that may combine to entail a propensity towards synaesthesia. This is referred to as “oligogenic inheritance” (Asher & alii 2009 in Marks 2011, 70). However, the fact that synaesthesia develops in an infinite number of ways, tends to prove that genetic factors cannot be solely responsible for the condition and that experiential and environmental aspects have to be taken into account (Marks 2011, 70).

14Along this line, other studies maintain that synaesthesia stems from “neurodevelopmental differences in the maturation of the brain among synaesthetes, which give rise to atypical ‘cross-talk’ between areas of the brain that would not normally interact” (Simner 23). This hypothesis is often referred to as “defective pruning”: “It arises […] from incomplete or defective pruning of connections between different but contiguous areas of the brain during development (which leads to a greater retention of connections that would normally be eliminated).” (Simner 23)

15Cambridge psychopathologist Baron-Cohen has proposed another explanation, compatible with the former, suggesting that there might have been a “modularity breakdown” hindering the disconnection of neuronal areas so that a “cross-talk” process also occurs. The latter “leakage theory”, just like the former, would have to do with anomalous brain maturation or defective apoptosis (neuronal pruning) involving some kind of neural over-connectivity (Baron-Cohen & alii 2007, 240).

16Grossenbacher and Lovelace make a different postulation which they term “disinhibited feedback” referring to the idea that sensory input travelling upward from one sensory mode to a higher more holistic multi-sensory centre, may in turn send the data processed back to its periphery according to a top-bottom movement, thus causing reciprocal multimodal sensory impressions due to a lack of downward inhibition. Taking into account those various explanations, different uncommon neural routes could therefore be involved in synaesthesia, either horizontally, in the case of defective pruning, or modularity breakdown, or vertically in the case of disinhibited feedback. It also lends support to the idea according to which “atypical neural wiring can produce radically different conscious experience” (Baron-Cohen & alii 2011, 240).

17The disinhibited feedback stance is more in keeping with those who claim that synaesthesia originates in other parts of the brain, i.e. in the limbic system for instance. This is the position adopted by Cytowic who came to this intuition because of synaesthesia’s affective and emotional overtones. The latter suggests that “parts of the brain get disconnected from one another […] causing the normal processes of the limbic system to be released, bared to consciousness, and experienced as synaesthesia” (Cytowic 1993 in Harrison 212). Other researchers have noted that, in the case of colour-grapheme synaesthetes, when the numbers were partly hidden, the actual colorful impressions only cropped up when close attention was paid to the figures, otherwise it did not (see Callaway), which points to activity in other cortical areas and undermines the idea according to which synaesthesia would occur automatically in primary sensory areas.

18An intermediate hypothesis within the nature-nurture discussion, is referred to as neural Darwinism: it casts light on the intricate reciprocal link between phenotype and genotype, i.e. between how our organism will develop various inherited features, morphologically and ethologically for instance, according to its surroundings and what it experiences; which is remarkable for it underscores the plasticity of the brain and its ability to adapt and counteract genetically inherited traits (Harrison 217).

19The aforementioned explanations all point to more or less innate factors. Yet the fact that it is also prominently linked to symbols that are culturally specific, and understood only by those who have become acquainted with them via intense learning processes, has also led to assumptions based on cognitive phenomena and learned associations; among which we find the idea that grapheme-colour or sound-colour synaesthesia might be due to children learning letters with artefacts like coloured refrigerator magnets, coloured letters on cubes or coloured alphabets (Whitthoft & Winaver 175). Although worthy of interest, this stance might be only partially acceptable, for if the letters took on the real hues of coloured cubes or magnets which children play with, then there would be far more consensus in the colour impressions depicted by synaesthetes; which is seldom the case, for synaesthesia is idiosyncratic and the chromatic impressions mentioned typically take on very specific and elaborate features:

Seers are invariably most minute in their description of the precise tint and hue of the colour. They are never satisfied, for instance, with saying ‘blue’, but will take a great deal of trouble to express or to match the particular blue they mean. (Galton qtd. in Harrison 111)

20Now, those cubes usually display very ordinary primary colours, which goes against synaesthetes’ sophisticated renditions of what they perceive.

21Another argument contravening the above assumption is that “consecutive letters may be closely related colours (e.g. ‘M’ = olive green, ‘N’ = emerald green, ‘O’ = washed out pale green). Coloured alphabet books go to great lengths to ensure that consecutive letters are printed in very different colours.” (Baron-Cohen & alii 2011, 239)

22Nonetheless, according to recent research notably in the field of anthropology of the senses (Classen 269), there is no denying that our cultural background may impact our perceptual system and that, although synaesthesia may obviously have genetic or neurological roots, it is likely to develop differently over time according to each individual’s cultural background and personal experience. The fact that there are never two perfectly identical types of synaesthesia lends support to this claim. Moreover, French semiotician Rosenthal underscores that, in other cultures, what he terms esthesic combinations might significantly differ from the usual forms of synaesthesia observed in our Western societies for the very reason that their sensory architecture is different from ours, with vision taking on a lesser importance for instance, and with more frequent pairings between auditory perceptions and smells, although Rosenthal acknowledges that more data is needed (2011a, 34-36). Likewise, the fact that so-called concurrents are very often of the visual type is probably due to our culture which places vision at the apex of the hierarchy of the senses (what Rosenthal coins oculocentrism [2011a, 36]). Accordingly, the fact that women are more prone to synaesthesia than their male counterparts, may be explained either genetically and/or socio-culturally, since gender difference is perhaps just as much a social-cultural construct as it is physiologically determined.

23I have not yet mentioned cases of synaesthesia that are artificially provoked by the intake of hallucinogenic drugs or those occurring after an accident has caused lesions in the brain; again those cases, thoroughly explored elsewhere in the literature, tend to prove that synaesthesia can be artificially initiated although it is still not clear whether here, it is of the same nature and displays the same origins as developmental synaesthesia.

Savant syndrome, mnemonics & literacy

24Now that we have underscored some of the difficulties one must wade through when trying to come to grips with synaesthesia, it is worthwhile addressing a very specific yet enlightening case. One in which synaesthesia is so vivid that not only does it spark off various sensory and emotional modalities instantly, but seems to pervade higher neurological levels thereby sporting a stronger conceptual dimension.

25The case is that of the famous British-born “synesthete-savant” with Asperger’s syndrome, Daniel Tammet, who has comprehensively depicted his condition in various writings. His testimony is of utmost value because of the multisensory clusters he experiences, leading to significant outcomes in term of data processing. For once, synaesthesia is not merely conjectured as some kind of worthless oddity, but seems to serve a purpose. Thus, considering the issue in reverse, by examining what full-fledged synaesthesia helps achieve cognitively, may enable us to a better understanding of its roots, mechanisms and purpose.

  • 1 Such as the famous mnemonist Solomon Shereshevsky (1886-1958) whose condition was thoroughly explor (...)

26What is most striking at first when examining the case of Tammet (or of any other autistic savant with extreme synaesthesia1) is his wondrous aptitude at mental arithmetic and his outstanding long-term memory, all of which seems to directly stem from his having developed strong synaesthesia after suffering a series of epileptic seizures (although his memory abilities are impaired when it comes to remembering faces and he scores normal capacities in terms of short-term memory [Baron-Cohen & alii 2007, 246]). When asked to mentally multiply a six-digit number with another six-digit number, which is rather challenging for any ordinary individual, he spontaneously visualizes these numbers in the guise of dynamic colorful three-dimensional shapes endowed with texture and personality even, at times, producing noises:

“The number two, for instance, is a motion, and five is a clap of thunder.” He adds: “When I multiply numbers together, I see two shapes. The image starts to change and evolve, and a third shape emerges. That’s the answer. It’s mental imagery. It’s like maths without having to think [...].” (Sheehan 104)

27Tammet sees the result unfold before his eyes as the form gradually becomes more distinct and literally embodies the arithmetical result. Furthermore, he has become famous for being able to recite the mathematical constant Pi to 22,514 decimal places for over five hours (Baron-Cohen & alii 2007, 241), which suggests that he is gifted with prodigious faculties in terms of memory. But for him calculating and reciting are not really a matter of painstakingly memorising a series of abstract digits by rote, but more about voicing what he sees when moving in some virtual landscape where digits visually display complex properties including number relationships, whether they are prime numbers or not, etc.:

I do not know why I think of 6 as tiny and 9 as very large or why threes are round and fours pointy. There are patterns, however, showing again that my shapes are meaningful and not random: 1 is bright, 11 is round and bright, 111 is round, bright and lumpy, 1,111 is round, bright, and spinning. This suggests […] that my brain has taken a small number of synaesthetic experiences for the smallest numbers and combined them in all kinds of ways to generate thousands of shapes – in the same way that languages take a small number of letters and sounds and generate thousands of words from them. (Tammet 191)

  • 2 We use the same technique to remember phone numbers by piecing them apart in two or three-digit chu (...)

28This is of course very useful in terms of mental calculation. Hence, by presenting numeral series in the more tangible garb of solid items standing for numeral chunks2, Tammet is able “to have very complex and determinate numerical representations of even very large numbers that carry information about the complex ways those numbers relate to other numbers.” (Matey 156)

29This bears considerable significance for it unveils some sort of rationale beneath the seemingly nonsensical phenomenon. Here indeed is exemplified the fact that if concurrents may vary in scope and amplitude, the forceful overpowering images induced can backfire and become meaningful and coherent. The supposedly incongruous concurrents elicited by digits, happen to coincide with other multisensory concurrents which in turn take us back to a more consensual representational form i.e. a digit corresponding to the arithmetical result (to which any one would have come to via more laborious means, like writing down digits and adding them one by one). Paradoxically, at the end of the process, it is the visual image that leads to the digit and not the reverse, which is not normally the case, for concurrents and inducers are hardly ever bidirectional (Mächler 5; for a full account of the phenomenon also referred to as implicit bidirectionality see Meier 127).

30Interestingly enough, a recent article written on the topic has revealed that brain areas involved in such astounding mnemonic techniques also referred to as mnemonisms, are linked to spatial recognition:

However, superior memory was associated with functional differences in brain areas usually involved in spatial processes and navigation. This might have been due to the use of strategies such as the method of loci which was applied by 90% of the superior memorisers. (Rothen & alii 1955)

31The loci method herein mentioned directly refers to the Greek poet Simonides of Ceos’s mnemonics, a technique whose outstanding posterity was thoroughly investigated by British historian Yates in her famous essay The Art of Memory, dating back to 1966. It consists in dismantling what is to be learnt, a speech for instance, into different parts, comparable to the “chunking” method mentioned above, and then placing those in different benchmark spots in a familiar place, a “palace of memory”. By mentally exploring the premises according to a specific order, one can easily retrieve the course of reasoning. In this respect, movement and visualization will significantly enhance memory, all the more so if associated with striking emotional imagery, according to the “classical memory advice on using emotionally charged images” (Yates 253). That of course is in keeping with the fact that Tammet’s numerals have their own personality or may emit “claps of thunder”. So here we find not only one type of synaesthesia but whole clusters of correspondences ranging from digit-colour/texture/sound to personification, kinetics and time-space synaesthesia where time or sequences are spatially encoded (Rothen & alii 1956). Whence it is easy to reflect that less cogent types of synaesthesia in which an a priori nonsensical additional form (colour, smell, flavor, movement, personality features) merges with the original percept, are but a milder expression of the manifold mental correspondences Tammet experiences spontaneously, perhaps due to the epileptic seizures he underwent and which might have amplified peculiar brain patterns and neural over-connectivity: “autism and epilepsy [and] schizophrenia might be related to reduced inhibition in the brain” (Tammet 187). As for people with ordinary mental capacities, they may have come to artificially resort to similar devices to improve their memory or better grasp difficult notions, thus endowing abstract, arduous concepts with figurative contours set in a familiar area: “[…] we remember more easily those things which are gross and sensible. And if we wish to remember intelligible notions more easily we should link them with some kind of phantasms, as Tullius teaches in his rhetoric.” (Yates 82)

32Presumably, the purpose of it all would be to weigh down what is otherwise too abstract, ephemeral or loosely diffused in or out of the body (like pain, or odors that are indeed very difficult to identify out of context) so that it may be sustainably recognized and embedded in our memory. Thus, signs that up until now were felt as too arbitrarily linked to their content, or paradigmatically involved in far too complex semiotic successions (like words referring to digits referring to numerical quantities, or words denoting musical notes encapsulating both pitch and sound), become “impregnated” with meaning when taking on a more familiar, easily recognizable natural aspect like a three-dimensional colorful shape. Revealingly, a recent article focusing on the obvious link between literacy and childhood synaesthesia posits the following statements:

[…] synesthesia normally develops as part of a difficult, formal process of learning to recognize and categorize perceptually and conceptually complex inducers. Even in cases of more unusual inducers, learning – especially perceptually and/or conceptually challenging forms of learning – seems to be at the heart of synesthesia. (Watson &alii 2014, 1)

33This analysis points to the fact that cognitive processes, either at a very early stage in the neonatal development, or later on, when having to learn letters, digits, numbers or even specific physical skills like swimming strokes (Watson & alii2014, 2), are facilitated when combining with concrete features such as colours or shapes. In that respect, synaesthesia would develop gradually as a “memory aid” in order to make sense of the outside chaos first perceived by infants (which echoes the defective pruning, apoptosis or modularity breakdown theories); or in order to help children decipher cryptic culturally-bound symbols such as those involved in reading, writing and counting. This is clearly in keeping with the fact that synaesthetes are more numerous amongst children than adults, and it also stresses the fact that it might not be an innate condition as such, but rather something that goes back to such an early period in post-natal life that synaesthetes believe their impressions were there all the while, i.e. as far as they can remember (which is not altogether false as it actually was concomitant with, if not a prerequisite to, the emergence of their personal memory). This is in keeping with Day’s list of types of synaesthesia, where what is made apparent is that the second term of the match tends to help categorizing, or fixing otherwise too abstract semiotic material (letters, chronological entities, months, days, etc.) or diffuse impressions like pain and odors; also, there might be a centripetal movement from a diffuse outside perception centering around a proprioceptive core (like the rare odor/flavor associations), or a reverse movement from something diffuse internally to a more easily delineated visualization projected outside of the body as in pain/vision associations.

34Ultimately, far from a confusion of the senses, synaesthesia would appear to hint at an elaborate mnemonic combination of abstract and concrete elements, merging to the point where their relationships become engrossed with meaning (although the process remains widely unconscious):

[…] concurrents represent more natural categories which are “just there” in the outside world and which can be comprehended without cultural education. If cultural artefacts (i.e., inducers) are more symbolic and less concrete than natural compared to concurrent categories (i.e., concurrents), then an inducer would activate fewer nodes compared to a concurrent (Collins and Loftus 1975). By this explanation, the association between inducer and concurrent is stronger and therefore the threshold for a conscious experience may be crossed more quickly. Reciprocally, the association between concurrent and inducer is weaker and therefore the threshold for a conscious experience may not be crossed. (Meier 130-131)

35Hence, idiosyncratic complex features that were initially but arbitrarily connected, come to be intrinsically linked; the more abstract or volatile element being weighted by a heftier one condensing various attributes in one sole material entity. In this way “complex information [becomes] available simultaneously.” (Paivio 9) This is also why writers of fiction would tend to make extensive use of metaphors and symbolic images. First, in order to put a name on what is but diffuse, erratic and surreptitiously felt internally hence mastering those potent feelings better; but also to “effectively encompass […] the gamut of features and associated ideas that define a general category [via] prototypical images” (Paivio thus mentions the image of the albatros in the famous poem “The Rime of the Ancient Mariner” by Coleridge (1797-1798), or that of the fin in Woolf’s novel, The Waves (1931), [Paivio 12-13])

Sensory cross-modality, art and creativity

36This leads us to considering the issue from a more imaginative vantage point. We will here envision synaesthesia no longer strictly as the singular condition which affects a few individuals, but as what Rosenthal, in the wake of Marks, terms inter-modalité i.e. the necessary sensory cross-modality which lies at the core of all perceptual and semiotic event (Rosenthal 2011a, 9; Marks 1975). My aim here is to show that, although the literary or pictorial effects also referred to as ‘synaesthesic’ may at first outwardly differ from the scientific accounts abovementioned, in fact they share common grounds in that they too operate a multisensory fusion of the symbolic and the mundane, crucial to the design and processing of expressive pieces as well as to the more basic handling of signs generally speaking (including those perceived in the outside extra-representational environment).

37To illustrate this, let us start with a simple pictorial case in point: that of the realistic visual rendition of a scene inspired by real events as exemplified in the well-known painting, The Raft of the Medusa by Géricault (1818-1819). Cognitive psychologist Solso’s analysis of the piece explicitly mentions the term “synaesthesia” to describe the manifold sensations conjured up:

While we do not “hear” the wind as it flaps the sail, or the waves as they splash around the raft, or even the cries for help as acoustic stimuli, we do certainly “hear” these things in our mind. The formal name for such a phenomenon is synesthesia, […] in which sensory information from one mode (such as visual sensation) psychologically activates another modality (such as an auditory sensation). As we look at the Raft, sight is primary but all other senses are psychologically active. We not only hear sounds but can also smell the sea and the putrefaction of decaying bodies, taste the salt air, and feel the cool sea water as it sprays over the raft. […] When we “see” this painting, the sensation is not confined to a single perceptual system but sweeps across many sensory modalities and psychological reactions to enrich the cognitive landscape. (Solso 4)

  • 3 He showed that perception implies sensory-motor reactions where one sensation can affect other perc (...)

38In this example, as in all figurative paintings generally speaking, the signs that are used mimic the visual qualities of their referents (contours, patterns, general shapes and composition, chromatic tonalities, hues, shades, etc.) The most emblematic visual features are extracted from a dynamic four-dimensional scene (three spatial dimensions and time) either being witnessed, imagined or recollected, and duplicated via the superimposition of oil soaked brushstrokes on the canvas, into a two-dimensional mute reflection of the scene where motionless figures stand in stark contrast against a blander background (according to well-known Gestaltism). From the very start, both the rendition of the scene and its treatment by a spectator require active selection and coherent homogenization of otherwise discrete and inane blurred units. Our faculties are actively solicited and firmly coordinated in order to retrieve just enough data to grasp the gist of the scene instantaneously, before a more in-depth interpretation. In our environment where everything complies with some spatiotemporal rule, things and beings are only ever perceived in an almost absolute fragmentary state, enshrined in the long string of causality that governs all. They are deprived of most of their dimensions, as we have no straight access to them externally (single perspective), internally (we don’t even have conscious access to part of our own internal representations and physical self), and in time (no immediate access to the thing or being’s future or past). To us, events or encounters become relevant solely when pointing to some kind of existence in relation to ourselves and our own Umwelt (a concept coined by German biologist Uexküll), i.e. as generating a whole set of qualia that are telling because they hint at potential fruitful intersubjective relations with our selves (for this reason letters may display various personality traits). In other words, what is perceived only makes sense to us if we can reach towards it by appending it either metonymically or metaphorically i.e. either by linking the part to the whole in both space (inferring the back from the front) and time (inferring the future from the past), or by matching a behavior with something relevant in our Umwelt. This is achieved thanks to sophisticated neuro-circuitry connecting various specialized cognitive areas, both horizontally and vertically, thanks to an “extensive parallel processing, which enables multiple processing of information” (Solso 115; also referred to as massive parallelism [Solso 119]), and also thanks to brain plasticity, adjusting to ever shifting stimuli by developing new connections. In this respect, active collaboration of the senses is a necessity and, instead of questioning the “unity of the senses”, to quote the title of Werner’s famous groundbreaking 1934 article3, one should focus on the reasons that make us think the senses “operate in isolation” (see Rosenthal 2011a, 26).

39An answer to this may lie in the recent research that led French neurophysiologist Berthoz to foster the concept of vicariance or vicariousness, whereby it is notably conjectured that our sensory system takes concurrent alternative vectors not only so that senses may supplement each other, but also to compensate for potential deficiencies (as in the case of temporary or permanent blindness for instance) and obtain confirmation of the data collected via different sensory routes (Berthoz 36). This means that the parts and the whole to which they pertain, intensely cooperate but can never totally merge; hence our persistent awareness of distinct sensory vectors. This is something that the process of simplexity, also devised by Berthoz, aptly demonstrates (14).

40Via complex mental simulations, all sensorial areas conglomerate, and re-ignition of past experiences naturally comes suffused with psychological and affective overtones. The brain thus spontaneously interprets what it perceives and anticipates what is to come for logical survival purposes. This logically leads to more abstract phenomena for what is envisaged goes beyond the scope of concrete utility goals, reaching for fruitful and inventive combinations and culturally enriched thoughts: “we reconstruct visual impressions, adding knowledge, interpretation, and bias to what we see and what it means.” (Solso 130) Not only is perception action but it elicits semiotic expression for we inject so much into what we perceive that it actually becomes an extension of our selves:

[…] we perceive objects as “directly expressing an inner form of life” (Werner, 1957, 69) that is, in the same manner in which we experience physiognomies, facial expressions, gestures, or, more generally, acts of living beings. Following this line, perceived forms are […] dynamic deployments, where the overall ‘dynamic tone’ is part and parcel of the experienced percept [Werner noted that colors are experienced not only in terms of hue, brightness, and saturation but also in terms of being strong or weak, cool or warm; lines not only have extent and curvature, etc., but may be seen as gay or sad]. Accordingly, all perceived objects, whatever their nature, partake of physiognomic qualities. (Rosenthal 2004, 229)

41Due to the demands of constant adaptation, mnemonic and predictive functions are closely entangled and lie at the core of any perceptive event, which allows for fertile semiotic invention paving the way for creativity. Interestingly enough though, recent research contends that real synaesthesia does not entail a higher degree of creativity (Ward & alli 139). As explained in the preceding lines, real synaesthesia is about “fossilized” associations (Watson & alli 2014, 1) stored in long-term memory at a very early stage in the child’s development and brain maturation, so that those connections are very unlikely to alter later in adulthood; they may only fall into oblivion (which is probably why there are more children with synaesthesia than adults). Nevertheless, this needs to be tempered, for synaesthetes, precisely because they often experience odd cross-modal combinations, may perform better at Remote Associates Test or RAT (Ward & alii 129-139); and may be more prone to creative art because they experience a richer world of representations (Meier 129). In the case of grapheme-colour synaesthesia, the reason why it could be incongruent with creativity surely lies in the fact that the fossilization process enables children to block spontaneous mental manipulations, and therefore avoid graphically confusing an N with a Z for instance, or a d with a b; all the more so as:

Unlike the identities of physical objects, the identities of letters are not invariant on rotation […] grapheme identity is perspective-relative, […] it is not possible to identify letters if one treats them as physical objects in space, subject to the conventions of standard shape recognition. Learning to see letters […] involves relearning many of the perceptual rules by which one identifies virtually any other shape in the world. (Watson & alii 2010, 6)

42Again, synaesthesia might be a way of inhibiting excessive endogenous imaginative processes by settling down forms once and for all. In this respect, artists greatly differ from synaesthetes because the former are mainly concerned with generating new forms rather than fixing old ones, the idea is not to settle but to unsettle things, establishing new links, abundant and transient multi-faceted associations rather than experiencing old, involuntary cross-modalities over and over again. Hence synaesthesia has greatly inspired artists obviously because what alleviates synaesthetes’ somehow parasitical neural activity, actually steered up their own imagination; all the more so as they had to figure out a way of expressing something they did not experience themselves, either directly or instinctively. For this reason, synaesthesia was often associated with such aesthetic creations that find a paramount expression in poetic metaphors or rich, symbolist poetry and sensuous pictorial renditions, whether abstract or figurative.

43In those cases, synaesthesia is no longer a matter of blocking neural over-connectivity but quite the reverse; it is about envisioning new paths whence the correspondences established are refined, readily inventive and achieved deliberately in view of being transcended in their turn. Swiss-German painter Klee aptly evokes this deliberate move forward beyond conventional signs and symbols, by inserting sober designed arrows, a recurring motif in his paintings. As it happens, Klee was an accomplished violinist, and is known to have played daily on his instrument before starting to paint, and willingly established strong analogies between geometric forms and acoustic vibrations (see such art pieces as Polyphony or Fugue in Red where polychromies and polymorphies combine to visually suggest polyphonies). Other artists have resorted to more drastic means in order to enhance their synaesthetic experience. Hence Baudelaire, whose poem “Correspondances” is systematically mentioned in the case of literary synaesthesia, seems to have made extensive use of hashish “a substance which he used in large doses over a protracted period of time” whereby his “synaesthesia ‘assumed an unaccustomed vividness’” (Harrison 116). As for Rimbaud and his equally famous synaesthesic poem “Voyelles”, we know that it was a sheer poetic invention for he also later confessed that he occasionally forced such poetic inventions upon his imagination (Harrison 119).

44To illustrate the way inventive associations can prompt thought-provoking multimodal sensory processes in their addressees along more abstract, cultural and affective connotations, we shall pay some attention to the Breakfast in Fur, a surrealist 1936 sculpture by German visual artist Oppenheim, where a small cup, a spoon and a saucer are intriguingly wrapped up in animal fur (a piece also analysed by Solso for other purposes [242]). What has been developed theoretically up above, is made obvious here, when an immediate repugnance takes hold of us as we inexorably imagine drinking tea out of this strange hairy item. Incongruous visual cues thus prompt a strange mixture of tactile and gustatory impressions soon tainted with sexual innuendo along with inevitable Freudian construal (see Solso 242). This is also due perhaps to the title’s proximity with Sacher-Masoch’s famous erotic novel Venus in Furs.

45There is no denying that synaesthesia taken in its widest acceptation, i.e. multi-sensory processes prompted by one sense or via the simultaneous stimulation of various sensory routes, always and necessarily involves not only a fusion of the senses, but expressive semiotic phenomena that cause the brain to extract meaning even from absurd and/or disturbing gratuitous aesthetic “polymodal associations” (Livingston 47).

Words

  • 4 The issue of synaesthesia necessarily raises that of subjective self-reports because, even when usi (...)

46If real synaesthetes and artists greatly differ in their way of apprehending multimodality, how about artists affected with real synaesthesia? The testimony of Russian-American novelist and synaesthete Nabokov, could prove most relevant to the issue at stake, for he seems to have consciously cultivated those vivid impressions, and willingly conveyed their potent significance to his readership via the “literarity” or “poeticity” he aimed at.4

47A few reminders before we move further. Words are graphically or acoustically emitted items that become meaningful only when associated with the signified component they conventionally and arbitrarily point to in the case of lexemes (Saussurean paradigm), or with verbal and pragmatic contextual cues in the case of function words (although when uttered alone, they do also display some kind of “bleached” meaning evocative of the type of syntactic link they serve to express). On the surface, the element conjured up displays no similarities whatsoever with the linguistic occurrence that triggered it, it is an endogenous image taken in the broadest acceptation of the term i.e. an internal impression whose semantic boundaries remain partly blurred, and may vary according to contextual cues, but whose core is supposedly shared by all speakers of a same linguistic community. Lexemes and function words encapsulate just enough characteristics to be understood but the way each person actually “comprehends” what is signified remains highly idiosyncratic (we all understand the word “dog” but the mental image conjured up as a response to the utterance or writing, takes on probably as many forms as there are speakers, and the same goes for grammatical items although the process is more abstract). According to what we developed at length earlier on, this bears strong resemblance with what real synaesthesia is all about, an apparently arbitrary association between an item felt as more concrete and one felt as more abstract, which become meaningful only when occurring jointly often along other related dimensions (movement, shape, physiognomic qualities).

48Subsequently, examining a small excerpt from Nabokov’s writings on the topic may prove useful all the more so as the novelist showed profound interest in the expressive, almost palpable dimension of words (see the famous opening lines of his most celebrated novel: “Lo-lee-ta: the tip of the tongue taking a trip of three steps down the palate to tap, at three, on the teeth. Lo. Lee. Ta. » [Nabokov 1955, 9]). In his autobiography, which originally blurred the boundaries between fictional and non-fictional prose (chapter 5, for instance, was first published in 1936, in the French literary review Mesures under the title “Mademoiselle O” [Nabokov 1951, 3]), he mentions his own synaesthesia in the following terms:

I present a fine case of colour-hearing. Perhaps “hearing” is not quite accurate, since the colour sensation seems to be produced by the very act of my orally forming a given letter while I imagine its outline. The long a of the English alphabet (and it is this alphabet I have in mind unless otherwise stated) has for me the tint of weathered wood, but a French a evokes polished ebony. This black group also includes hard g (vulcanized rubber) and r (a sooty rag being ripped). Oatmeal n, noodle-limp l, and the ivory-backed hand mirror of ‘o’ take care of the whites. I am puzzled by my French on which I see as the brimming tension-surface of alcohol in a small glass. Passing on to the blue group, there is steely x, thundercloud z, and huckleberry k. Since a subtle interaction exists between sound and shape, I see q as browner than k, while s is not the light blue of c, but a curious mixture of azure and mother-of-pearl. Adjacent tints do not merge, and dipthongs do not have special colors unless represented by a single character in some other language (thus the fluffy-gray, three-stemmed Russian letter that stands for sh, a letter as old as the rushes of the Nile, influences its English representation). (Nabokov 1951, 58)

49The genius of a polyglot writer gifted at reminiscing and articulating his innermost perceptions and feelings, is of great value to the present analysis for it emblematizes the issue as to whether remarkable literary acumen can be as enlightening as its hard scientific counterpart. Nabokov’s rendition of what he experiences thus thwarts the usual colour-hearing simplistic definition. It hints at the inherent qualities mentioned before, where there is no strict ontological boundary between inducer and concurrent and where shape, hues and texture meaningfully merge. His account reinforces the idea that concurrents scarcely display only one dimension and that colours rise along with shapes (“long a”), kinesthetic features (“limp”), graphic ones (“thundercloud z”), sounds linked to exertion of some sort (“a sooty rag being ripped”), as well as textures, the word “rubber” instantly conveying the idea of direct skin contact. Such amalgamation probably stems from the same intricate physio-cognitive workings that lead to the emergence of meaningful verbal forms, what enactive linguists now refer to as “languaging”, which occurs any time a word is emitted externally (“extraverted vocal languaging”), internally (“introverted vocal languaging”) or graphically (“extraverted manual languaging”) (Bottineau 2008a, 8). Thus, the internal representations of a string of letters, just like that which is triggered by words, mechanically imply mute sensory-motor activation of the tongue, teeth, lips and gums (all exapted from their original functions - lapping, eating, biting, chewing, sucking, etc. - due to selective adaptation [Bottineau 2012, 81]), but also of mental kinetic impressions of the arms, hands and fingers involved in the act of drawing. The “hand mirror” visually reminds us of an oval silvery object comparable to an o, much like what a mirror does in fact; it can also be interpreted in reverse as suggesting an act of mirroring with the hand, while the “noodle-limp” personified l semantically embodies the graphic looseness of its loop. Let us not forget that the learning of letters is often associated with their cursive rendition on paper (a strenuous task for children whose movements are not yet fully coordinated), a synaesthesic process common to all of us:

Recent neural imaging has revealed unexpected connections between perceptual and motor areas even in cases when vocal production is not involved, and recent studies in infants have shown these connections to be active long before the articulatory system is operational: whether one does or even can speak or not, the processing of acoustic signals in linguistic terms is correlated with the controlling of the muscles that will produce them. (Bottineau 2008a, 16)

50Here, sensory input and motor output are one and the same, a fusion also termed perçaction by Berthoz, debunking the usual passive/active dichotomy linked to perceptual events, and showing in what way letter and word perception whether auditory, graphic or visual always involves the active echoing of what is perceived holistically i.e. by re-enacting all their possible means of production. This is done not only by experiencing anew what is being formulated via introverted mechanical duplication, but also by reminiscing the lengthy processes behind language acquisition. It is perhaps therefore not a surprise that the words associated with these visual impressions also trigger sensuous childhood memories: “oatmeal n” encapsulates the gustatory and olfactory sensations of children’s food; just as noodles, soft and warm like the tongue gently bouncing on the alveolar area to produce the /l/.

51The extract also aptly illustrates the fact that alphabetically adjacent strings of letters, because they are learnt in strict linear order, tend to share common hues along with some kind of colour gradient: there is the “black group”, that of the “whites” and, finally, that of the blue. All this modality of symbolic preserves the above-mentioned simplexity principle theorized by Berthoz, where letter discrimination is secured via a colour association that does not preclude a feeling of overall unity (“It seems likely that it is easier for children to hold sequences of colours in short term memory than sequences of monochrome letters.” [Watson & alii 2010, 8]).

52Nabokov learnt how to read in English before he did in his own mother tongue and learnt both English and French at the age of six [Nabokov 1951, 57]). This means that he underwent the very demanding task of learning how to read and write in three different languages almost simultaneously. Due to the languaging processes described in the preceding lines, the novelist naturally dissociates the English “long a” from its French equivalent (a letter which probably also triggers a Russian phonetic impression of the type /a:/, the letter a being common to both Roman and Cyrillic scripts). Because of automatic internal voicing, a choice has to be made for one equally acquainted with various tongues. This supports the argument for the cognitive interpretation that considers synaesthesia as a learning aid occurring at a specific time in the course of a child’s education and brain maturation, when the brain is still sufficiently plastic to undergo certain “leakages” (Watson & alii 2014, 3). It does not contradict the genetic hypothesis mentioned before (Nabokov’s mother was also a synaesthete [Nabokov 1951, 59]). It is as if this cognitive skill were due to a genetic predisposition prompted by environmental factors.

53Then Nabokov goes on to question the hues of two close related phonemes embodied by the letters s and c, or that of q “browner than k”. Why should one be darker than the other? Perhaps because of the heftier impression the sound generates, sound pitch being somehow related to a comparable degree of luminance. Here again the idea that those associations are totally arbitrary is challenged. Those singularities point to some kind of common iconic quality shared by sound and vision. As discussed earlier when evoking Werner’s famous article on the “unity of the senses”, there is evidence suggesting that sound perception can distort visual perceptions when they occur together and vice versa. Moreover, recent neurological research backs the hypothesis that degree of luminance and digit magnitude are intrinsically linked in both synaesthetes and non-syanesthetes, because “in our cortical organization and mapping […], the maps linking magnitude are inextricably linked” (Cohen-Kadosh & alii 835), hence the title of the scientists’ article: “Small is bright and big is dark in synaesthesia.”

54What is particularly striking in this account is that most colours mentioned are not designated by mere ordinary adjectives but very often by complex compound adjectives, nouns or even complex nominal phrases. Those minute descriptions are prototypical of synaesthetes. Yet, here the phenomenon reaches heights. The phrase “colour-hearing” proper is a “fine case” of compound word formally speaking (to misdirect Nabokov’s own words). It associates a noun (the modifier) and a verb (the head) with the –ing marker hinting at some kind of active process (according to the natural syntactic impact of the left element on the right one in English). It is also a conventional phrase used as an approximation to refer to the condition. Indeed, in the first accounts of the phenomenon by 19th century Frenchman Flournoy, colour-hearing stood paradigmatically for all types of synaesthesia. The phrase is therefore a hyponymic expression referring to a prototypical manifestation of it. Yet, out of this common, rather inaccurate basic compound, Nabokov forges an impressive wealth of other compounds as if responding to some kind of inventive bid, stretching the limits of what restricted linear verbal means can semantically achieve. He thus goes through the whole gamut of word associations ranging from past participle adjective + noun (“weathered wood”, “polished ebony”, “vulcanized rubber”), to the reverse solution with “ivory-backed”, to the looser noun + noun association with a space: “polished ebony”, a hyphen: “three-stemmed”, or the tighter directly appended solution as in “oatmeal”, not to mention more elaborate analytical nominal phrases such as “curious mixture of azure and mother-of-pearl” that connect a fixed expression to a more unusual idea “curious mixture of azure” (whereby we move from a two-dimensional verbal emission to a multifaceted structure echoing synaesthesia’s complexity).

55He also aptly sheds light on the fact that similar syntactic appositions may give rise to various semantic treatments. The basic meaning may lie in the second term while the first only modifies it (as in “fluffy-gray”, see the modifier/head schema discussed above), but also lead to a more opaque situation where one or both elements have lost their original meaning as in “huckle” in “huckleberry” (“huckle” etymologically comes from ‘hip, haunch’ because of the plant’s jointed stems” [OED 1999, 691], an interesting illustration of the role of physiognomy in words’ genesis; for an extensive analysis of the phenomenon see Argoud). Sometimes, both entities’ strict semantic values are preserved as in “tension-surface”, suggesting a threat of imminent spillage.

56The fact that instinctive discrepant associations may nurture new connections and ignite inventiveness is impeccably illustrated. These are all compounds enabling a precise and refined verbal rendition of subtle multimodal impressions (“fluffy-grey”), but also, perhaps, metalinguistically referring to the stunning way in which language works by splitting linearly what is perceived holistically and then rearranging new, compelling semantic ensembles.

57This stimulating piece of inventiveness would require a more in-depth analysis, unfortunately not possible here. Let us just ponder a while on the following: the thought-provoking succession of “Oatmeal n, noodle-limp l, and the ivory-backed hand mirror of o”: “oatmeal n impresses us as a reshuffling of the normal MNO order just like, on a higher scale at the paragraph’s level, the strange NLO sequence chosen by Nabokov who starts with oatmeal N, noodle-limp L and concludes on the hand-mirror O. In the middle occurrence, the grapheme o is duplicated in the word evoked by the l, “noodle-limp”, thus turning its phonetic rendition into a long /u:/ rather than the original /oʊ/, graphically duplicating, phonetically dilating and distorting the sound. Oddly enough, while, logically, the /l/ reverberation is achieved via a visual and acoustic recurrence of ls in the expression “noodle-limp l”, there is no /n/ in oatmeal, just the letters that alphabetically frame it but in reverse: O then M, as if the outside were backwardly pointing to an invisible inside. A reverse phenomenon follows with the reverberated consonance /l/ in “noodle” and “limp”, first at the end of the first word then at the beginning of the last. The word “limp” enshrines the following /m/ sound as if weighing it down more securely and enabling it to resonate better, just as colours are better understood when embodied in some kind of emblematic object. All in all, the original LMNOP string comes to life via an apt reshuffling of those entities inserted in superior semantic items that bear a consonantic value capable of reverberating physically, cognitively and semantically what these forms actually suggest to a synaesthete and a creative writer: a symphonic re-orchestration of itself.

58If immediate sensory experience can trigger multimodal phenomena of the highest complexity synchronically, interestingly enough, synaesthesia can also be detected in the way linguistic features have evolved throughout history, diachronically. We have very briefly alluded to this above when evoking the etymology of the word “huckleberry” where “huckle” stems from a physiognomic analogy with our own physical joints (hips, knuckle). And Tammet’s illuminating remark hinted at earlier also points to “savant numerical abilities [as being] comparable in complexity to language abilities in most people.” (Tammet 186)

59But the phenomenon goes beyond what one could expect. Recent research in linguistics shows that the emergence of semantic signifiers has gradually evolved along intricate iconic processes where some of the sounds that make up words partly mimic multimodal tensions caused via interaction with the outside world. Those micro-semantic forms have been termed “phonaestemes”, originally coined by English linguist Firth in 1930, and, more recently, “submorphemes” by English linguist Philps or “cognemes” by French linguist Bottineau. The latter has drawn a list of semantic consonantic clusters that act as core invariants in the English lexicon, i.e. “clusters of more than one phoneme, and occasionally isolated phonemes, which appear to relate the lexical notion to a specific semantic field.” (Bottineau 2008b, 21). I will mention but a few of those listed by Bottineau (2006, 311): -sk in mask, skin, scratch, scrub, skate, etc. (idea of surface), the famous gl- as in glimmer, glitter, glisten, etc. (referring to luminosity), or tw- in two, -tween, twin, twitch, etc. (binarity).

60Concerning the cluster gn-, Philps shows that it can be found in many words referring to the action of the jaws:

<g> in <gn-> may originally have functioned as a ‘core invariant’ and <n> as a variable in the prehistory of English, specifically in Proto-Indo-European. Analysing gn- in most ‘gn- words’ of Anglo-Saxon origin and some of Greek origin as the surface trace of an underlying notional domain linked to jaw-related phenomena (sound articulation, mastication, manducation, etc.) suggests that there exists an extralinguistically motivated, causal relation between <gn-> and this domain involving the concept of ‘submorphemic iconicity’ (Philps 125)

61Obviously, the production of those sounds can remain subconsciously meaningful to any contemporaneous speaker because some of them still correspond to some kind of physical process like clenching the jaws in manducation.

62But those entities can also undergo interesting synaesthetic processes in their turn. This is exemplified by the word “chroma” (χρῶμα, khrỗma) whose meaning would also originate in Proto-Indo-European “ghreu”: “to grind”, “to rub” (Letalleur-Sommer 69), a word whose etymology takes us back to skin complexion and, secondarily, to touch, then to colour and, eventually, to music semi-tones:

Quality or intensity of color, 1889, from Gk. khroma ‘surface of the body, skin, color of the skin, color,’ related to khros "surface of the body, skin," khrozein "to touch the surface of the body, to tinge, to color," explained as being somehow from Proto-Indo-European *ghreu- ‘to rub, grind’ (see grit (n.))5 (see Online Etymology Dictionary).

63Touch, vision and hearing are here closely entangled due to sensory-motor processes (action of rubbing and grinding). Thus, with time, this or that modality may be focused upon whilst others remain in the background but multimodality is always present. Word emergence phenomenologically derives from mimicking in praesentia what is perceived via sensory-motor processes. Later, the re-enacting of the sensory-motor action in absentia, will become an iconic signifier. A gradual move away from the original meaning of these forms and their insertion in superior semantic entities, will further occasion a movement from original iconicity towards apparent arbitrariness.

Conclusion

64Synaesthesia proper seems to have various origins ranging from genetics to anomalous cross-modalities and neuronal connections, notably between those responsible for syntactical organization (Broca’s area) and those that have to do with “sequential and logical spatial abilities (left parietal lobe)” (see Tammet 18). For those reasons, it would be triggered at a peculiar neuro-physiological stage in the development of a child’s brain, for cognitive purposes. Yet, neural over-connectivity would only embody a paramount expression of something likely to occur anyhow in anyone to enhance syntactical and linguistic skills, thus giving more substance to human productions that leave no trace elsewhere than in the mind, hence its link to the more concrete sensory-motor writing processes. This would be achieved in a way that is probably not arbitrary, but idiosyncratic and analogical, varying according to each individual’s imagination and past experiences (much like the image of the dog earlier mentioned). The process would not hinder communication because the sounds and forms produced in both extraverted vocal and manual languaging, contain enough shared features to be understood.

  • 6 The expression “ghostly form” is a reference to Didi-Huberman’s study L’image survivante, inspired (...)

65Consequently, synaesthesic phenomena openly challenge linguistic arbitrariness, for any languaging process contains remnants of old analogies combining various sensorial modalities, although this is established according to each and everyone’s logic, personal history and sensitivity. In this respect, synaesthesia could be a pointer to a larger modality of symbolic composition, vital to the building of meaningful life experiences at large as well as learning aids. At the moment of birth, those would enhance differentiation between various sensations, then enable the discriminative processes involved in the emergence of simple ontological categories and, lastly, they would confer meaning to cryptic cultural intersubjective inventions such as musical notes, letters, or numbers, perceived as totally arbitrary by young neophytes. Eventually, in adult life, the process could be re-ignited artificially via new imaginative processes reshuffling the semiotic systems that had been fixed by conventions, thus giving a new lease of life to old “ghostly forms” fallen into oblivion6.

Top of page

Bibliography

WORKS CITED

ARGOUD, Line. Corps, conceptualisation, émergence du sens. Etude lexico-cognitive de trois segments consonantiques en anglais. Saint-Etienne: Publications de l’Université de Saint-Etienne, 2012.

ASHER, Julian E., LAMB, Janine A., BROCKLEBAND, Denise, CAZIER, Jean-Baptiste, MAESTRINI, Elena, ADDIS, Laura, SEN, Mallika, BARON-COHEN, Simon, & Anthony MONACO. “A Whole-Genome Scan and Fine-Mapping Linkage Study of Auditory-Visual Synesthesia Reveals Evidence of Linkage to Chromosomes 2q24, 5q33, 6p12, and 12p12.” American Journal of Human Genetics 84, n° 2, (2009): 279-285.

BARNETT, Kylie J. & alii. “Familial patterns and the origins of individual differences in synaesthesia.” Cognition. vol. 106, n° 2, (February 2008): 871–893.

BARON-COHEN, Simon, BOR, Daniel, BILLINGTON, Jac, ASHER, Julian, WHEELWRIGHT, Sally, & Chris ASHWIN. “Savant memory in a man with colour-form-number synaesthesia and Asperger syndrome.” Journal of Consciousness Studies 14 (9-10), (2007): 237-251.

—. BURT, Lucy, SMITH-LAITTAN, Fiona, HARRISON, John, & Patrick BOLTON. “Synaesthesia: Prevalence and familiality.” Perception 25 (9) (1996): 1073–1080.

BAUDELAIRE, Charles. Les Fleurs du Mal. 1857. Paris: GF, 2008.

BERTHOZ, Alain. La Vicariance: le cerveau créateur de mondes. Paris: Odile Jacob, 2013.

BOTTINEAU, Didier. « Le langage représente-t-il ou transfigure-t-il le perçu ? », Formes sémantiques, langages et interprétations : Hommage à Pierre Cadiot, n°spécial de La TILV (La Tribune Internationale des Langues Vivantes), F. Lautel-Ribstein (éd.), Perros-Guirec: Anagrammes, 2012: 73-82.

—. “Language and Enaction.” Enaction: towards a new paradigm for cognitive science. Stewart, J., Gapenne, O. & Di Paolo, E. Eds. Cambridge, Mas.: MIT (2008a): 1-67.

—. “The submorphemic conjecture in English: towards a distributed model of the cognitive dynamics of submorphemes”. Lexical Submorphemics. Submorphémique lexicale. Lexis, (2008b): 19-42.

—. “L’émergence du sens par l’acte de langage.” M. Banniard & D. Philps. La fabrique du signe, Linguistique de l'émergence. Toulouse: Presses Universitaires du Mirail, 2006: 299-325.

CADIOT, Pierre. “Schematics and Motifs in the Semantics of Prepositions.” Prepositions in their Syntactic, Semantic and Pragmatic Context. Amsterdam: John Benjamins, (2002): 41-57.

CALLAWAY, Ewen. “Pay attention for synaesthesia to emerge.” New Scientist, vol. 204, (2009): 6.

CLASSEN, Constance. “Other ways to wisdom: learning through the senses across cultures.” International Review of Education – Internationale Zeitschrift für Erziehungswissenschaft– Revue Internationale de l’Education 45 (3/4), (1999): 269–280.

COHEN-KADOSH, Roi, HENIK, Avishai, & Vincent WALSH. “Small is bright and big is dark in synaesthesia.” Current Biology vol 17, n°19, (2009): 834-835.

CUPCHIK, Gerald C., VARTANIAN, Oshin, CRAWLEY, Adrian, & David J. MIKULIS. “Viewing artworks: Contributions of cognitive control and perceptual facilitation to aesthetic experience.” Brain and Cognition 70, (2009): 84-91.

DAY, Sean A. “Synesthesia: demographic aspects of synesthesia.” www.daysyn.com/Types-of-Syn.html

DIDI-HUBERMAN, Georges. L’Image survivante. Histoire de l’art et temps des fantômes selon Aby Warburg. Paris: Minuit, 2002.

DIXON, Mike J., SMILEK, Daniel, & Philip M. MERIKLE. “Not all synaesthetes are created equal: Projector versus associator synaesthetes.” Cognitive, Affective, & Behavioral Neuroscience n° 4 (3), (2004): 335-343.

FIRTH, John Rupert. The Tongues of Men & Speech. 1930 & 1937. Oxford: Oxford University Press, 1964.

FLOURNOY, Théodore. Des phénomènes de synopsie (audition colorée). Paris: Alcan, 1893.

GALTON, Francis. “Visualised numerals.” Nature 21, (1880): 252–256.

GERICAULT, Théodore. The Raft of the Medusa. Oil on canvas. 1819. Musée du Louvre, Paris.

GROSSENBACHER Peter G. & Christopher T. LOVELACE. “Mechanisms of synesthesia: cognitive and physiological constraints.” Trends in Cognitive Sciences vol. 5, (Jan. 2001): 36-41.

HARRISON, John. Synaesthesia. The Strangest Thing. Oxford: Oxford University Press, 2001.

HARRISON, John & Simon BARON-COHEN. “Synaesthesia: reconciling the subjective with the objective.” Endeavour 19(4), (1995): 157-60.

HUBBARD, Edward & Vilayanur S. RAMACHANDRAN. “Neurocognitive Mechanisms of Synesthesia.” Neuron vol. 48, (Nov. 3 2005): 509-520.

KLEE, Paul. Polyphony. Tempera on linen. 1932. Emanuel Hoffman Foundation, Kunstmuseum, Basel.

KOHLER, Wolfgang. Psychologie de la forme. 1929. Paris: Folio, 2000.

LETALLEUR-SOMMER, Séverine. “Meaning Construction and Colour Schemes: an enactive paradigm.” Definitions of Color/de la Couleur. F. Ogée & M. Géracht (eds). Interfaces - image, texte, langage vol. 33, (2013): 61-80.

LIVINGSTONE, Margaret. Vision and Art. The Biology of Seeing. New York: Abrams, 2002.

MACHLER, Marc-Jacques. “Synaesthesia and learning. A differentiated view of synaesthetic perceptual awareness.” Zurich, 2009. www.synesthesia.com

MARKS, Lawrence E. “Synaesthesia, Then and Now.” Intellectica n°55, (2011/1): 47-80.

—. “On colored-hearing synesthesia: Cross-modal translations of sensory dimensions.” Psychological Bulletin 83, (1975): 303-331.

MATEY, Jennifer. “Can Blue Mean Four?” Sensory Integration and the Unity of Consciousness. Ed. David J. Bennett and Christopher S; hill. Cambridge, Mass.: M.I.T. Press, 2014: 151-170.

MEIER, Beat. “Semantic representation of synaesthesia.” Theoria & Historia Scientarum vol. 10, (2013): 125-134.

NABOKOV, Vladimir. Speak, Memory. 1951. London: Penguin, 1967.

—. Lolita. 1955. New York: Random House, 1997.

ONIANS, John. Neuroarthistory. From Aristotle and Pliny to Baxandall and Zeki. New Haven: Yale University Press, 2007.

OPPENHEIM, Méret. Breakfast in Fur. Assemblage sculpture. 1936. Museum of Modern Art, New York.

PAIVIO, Allan. “The Mind’s Eye in Arts and Science”. Poetics 12, (1983): 1-18.

PHILPS, Dennis. “Submorphemic iconicity in the lexicon: a diachronic approach to English –gn”. Lexical Submorphemics. Submorphémique lexicale. Lexis, 2008: 125-139.

RAMACHANDRAN, Vilayanur S. & Edward M. HUBBARD. “Synaesthesia – a window into perception, thought and language.” Journal of Consciousness Studies 8, (2001): 3-34.

RIMBAUD, Arthur. Œuvres, Paris: Garnier, 1991.

ROSENTHAL, Victor. “Synesthésie en mode majeur. Une introduction.” Synesthésie et intermodalité. Intellectica 55, 2011a: 7-46.

—. “Amalgame des sens, ivresses divines, chromatismes : documents et témoignages.” Synesthésie et intermodalité. Paris: Intellectica 55, 2011b: 171-197.

—. “Microgenesis, immediate experience and visual processes in reading.” Ed. A. Carsetti. Seeing, Thinking and Knowing: Meaning and Self-Organisation in Visual Cognition and Thought, Kluwer Academic Publishers, 2004: 221-243.

ROTHEN, Nicolas, MEIER Beat & Jamie WARD. “Enhanced memory ability: Insights from synaesthesia.” Neuroscience and Biobehavioral Reviews 36, (2012): 1952–1963.

SACHER-MASOCH. Venus in Furs. 1870. London: Penguin, 2000.

SAGIV, Noam & Jamie WARD. “Crossmodal interactions: lessons from synesthesia”. Progress in Brain Research vol. 155, (2006): 259-271.

SHEEHAN, William. A Passion for the Planets. Envisioning other Worlds from the Age of the Pleistocene to the Age of the Telescope. New York: Springer, 2010.

SIMNER, Julia. “Beyond perception: synaesthesia as a psycholinguistic phenomenon”. Trends in Cognitive Sciences vol. 11 n°1 (2006): 23-29.

SOLSO, Robert L. The Psychology of Art and the Evolution of the Conscious Brain. Cambridge, Mas.: MIT Press, 2003.

TAMMET, Daniel. Embracing the Wide Sky. A Tour Across the Horizons of the Mind. London: Hodder & Stoughton, 2009.

UEXKÜLL, Jakob Johann von. Mondes animaux et monde humain, suivi de Théorie de la signification. 1909. trad. de P. Muller. Paris: Denoël, 1965.

WARD, Jamie, Daisy, THOMPSON-LAKE, Roxanne, ELY & Flora KAMINSKI. “Synaesthesia, creativity and art: What is the link?” British Journal of Psychology 99 (2008): 127-141.

WATSON, Markus R., AKINS, Kathleen, SPIKER, Chris, CRAWFORD, Lyle & James T. ENNS. “Synaesthesia and learning: A critical review and novel theory.” Frontiers in Neuroscience vol. 8, 98 (2014): 1-15.

WATSON, Markus, R., AKINS, Kathleen & Lyle CRAWFORD. “The developmental learning hypothesis of synaesthesia: A summary.” Studiez aplikované lingvistiky/Studies in Applied Linguistics 1(1), (2010): 1-14.

WERNER, Heinz. “L’unité des sens” (1934). Synesthésie et intermodalité. Paris: Intellectica 55, (2011): 159-170.

WITTHOFT, Nathan & Jonathan WINAWER. “Synesthetic Colors Determined by Having Colored Refrigerator Magnets in Childhood.” Cortex; a journal devoted to the study of the nervous system and behavior 42(2), (2006): 175-183.

WOOLF, Virginia. The Waves. 1931. London: Wordsworth, 2000.

WORDSWORTH, William & Samuel Taylor COLERIDGE. Lyrical Ballads. 1798. New York: Routledge Classics, 2005.

YATES, Frances. The Art of Memory. 1966. London: Pimlico, 1992.

Top of page

Notes

1 Such as the famous mnemonist Solomon Shereshevsky (1886-1958) whose condition was thoroughly explored by Soviet psycho-neurologist Alexander Luria; (Rosenthal 2011b, 171-176).

2 We use the same technique to remember phone numbers by piecing them apart in two or three-digit chunks.

3 He showed that perception implies sensory-motor reactions where one sensation can affect other perceptual areas for instance when the hearing of a sound may alter how we simultaneously perceive a colour. (Werner 169)

4 The issue of synaesthesia necessarily raises that of subjective self-reports because, even when using MRI techniques, it is difficult to see what is going on in the brain, so neuro-psychologists have to base at least part of their research on synaesthetes’ accounts. (Harrison & Baron-Cohen 160). In this respect, clinical synaesthesia and its literary rendition by writers may greatly differ in that the second tends to add an “extra literary layer” to the original experience for aesthetic purposes.

5 http://www.etymonline.com/index.php?allowed_in_frame=0&search=chroma&searchmode=none. Last consulted on January 11, 2015.

6 The expression “ghostly form” is a reference to Didi-Huberman’s study L’image survivante, inspired by Aby Warburg “Mnemosyne-Atlas”, where it is shown that images live, as it were, a life of their own away from their original significance, which challenges the apparent immediacy of visual representations.

Top of page

References

Electronic reference

Séverine Letalleur-Sommer, « More than a condition: an examination of synaesthesia as a key cognitive factor in the processing of reality and in its literary and pictorial renditions », Interfaces [Online], 36 | 2015, Online since 12 February 2018, connection on 24 March 2019. URL : http://preo.u-bourgogne.fr/interfaces/index.php?id=230

Top of page

About the author

Séverine Letalleur-Sommer

Université de Paris-Ouest la Défense

Top of page
  • Logo Université de Bourgogne
  • Logo College of the Holy Cross
  • Logo Université Paris Diderot - Paris 7
  • Logo CNRS INSHS
  • Logo Centre Interlangues TIL
  • Logo LARCA
  • Logo MSH Dijon