'From Image to Text' by Bill Hart
The spoken and written word are generally thought of as two related aspects of the one thing – language. However, images are thought of as belonging to a different order of communication. In recent years, I’ve become fascinated by the process through which images become symbols, not just the evolution of the graphic form, but the associated distillation of an ill-defined thought into a concept which can be labelled with a word or symbol and freely communicated. In this article I make the case that the written word may have a closer connection to imagery than it does to speech.
'Faith in Reason' Bill Hart, 2008What’s it like to be a painting? What a question. The answer has to be: probably not much. But the point of asking such a question is to prompt the thought that perhaps certain works of art do have the property of being in a special way “like something,” a something that again is very hard to capture in words.
Allow me a minute to play with just this thought. Suppose there is an analogy between a work of art and “a work of sensation,” where can we go with it? Well, to start with, we might want to use artistic methods and media instead of ordinary language as analytic tools for exploring the nature of phenomenal experience.1
Phycologist / philosopher Nicholas Humphrey’s theory of consciousness, which like much recent thought on mind and consciousness, rehabilitates emotion and sensation as the key elements of thought. Humphrey considers art as analogous to the sensation of consciousness. Since the Enlightenment, symbolic logic and reasoning have been placed firmly at the apex of human intelligence, but theories such as those of Humphrey’s, based on research findings in neurophysiology and cognitive science, call for a re-evaluation of the relationship between symbols and images in human thought and expression.
For much of the past century an assumption of linguistics, psychology and cognitive science has been that we think in code. Words and symbols are received by the human sensory apparatus of eyes and ears and translated by the organic computer into ‘mentalese’ - the language of thought.2 The symbols from the physical environment, are translated into the symbols of the mental world, processed and acted upon. From this view, symbols become the essence of thought. Imagery, because it can’t be directly translated to a neat set of symbols, sits uncomfortably outside this model. Symbols have assigned meanings, and arranging symbols allows more complex meanings to be encoded, but in this symbol processing view of mind, the perception of meaning is deeply mysterious.
Over the course of human evolution there have been three great innovations in cognition: speech; drawing; and the computer. The first had no dependence on technology. The second is dependent on technology (mark making tools and surfaces) and encompasses not only the ability to communicate independently of time and presence, but also encompasses pictorial and symbolic thought, mathematics, writing, and representation. The third, the computer, is the ultimate symbol-processor, made manifest through the most precise application of craft and technology we have ever known.
As speech and language gave us the ability to communicate our inner states of mind, and must have led to the internal theory of mind that enables us (sometimes) to think and feel as others, to ‘know’ their mind, so did drawing and writing allow the compilation and codification of experience and knowledge. The computer has given us the ability to further explore the limits and test some of the assumptions of that knowledge, and from this exploration, new insights into the nature of thought and language have started to emerge in recent years. In particular, the failure of Artificial Intelligence researchers to develop a ‘thinking machine’ has caused a re-evaluation of logic and symbolic processing as the engines of the mind.
The technology of surface and marker, pencil and paper is one of the most flexible and powerful of thinking tools; it is hard to imagine a means of representation more suited to enable the transition from vague sensation to abstract thought, nor one that can allow expression of ideas in such a diversity of forms from pictorial to symbolic. Drawing enables cognition that expands beyond the serial moment of one thought-image followed by another. Marks act as placeholders for ideas, the two-dimensional space of paper allows many more and even contradictory thoughts can be simultaneously arranged in different geometries. Today, we tend to think of drawing as the sole province of children and artists, without considering the lists we scrawl, the doodles we do when bored, the crude sketchy diagrams we make to explain something or to just ‘figure something out’, as drawings.
'Communal Sense' Bill Hart, 2008Visual art is the process of thinking through making, or understanding the world via creating representations. It is, in essence, a process of constructing analogy. Considered in the light of these emerging understandings of cognition, visual art is not inherently inferior to other cognitive processes for gaining and communicating knowledge. Art historian and theorist Barbara Stafford argues, in Visual Analogy: Consciousness as the Art of Connecting, for a rehabilitation of analogous reasoning from the dominant allegorical mode of postmodernism:
I want to recuperate analogy, then, as a general theory of artful invention and as a practice of intermedia communication. Knowledge is a heuretic system always in pursuit of equivalences for one thing or another. It results when abstractions are made concrete, when family ties between distant or separated events are exposed.3
Following on from drawing, the development of first spoken language and subsequently written language can be argued to be among humanity’s most powerful technological achievements. Symbolisation, the process by which images become symbols, is so effective that we are tempted to think of it as transparent. Consider letterforms; for us they no longer have a pictorial significance; we are able to almost instantly recognise a vast variety of variations of the same letterform as that symbol. We have become so adept that it is tempting to conclude that the ability is innate. We are trained in the ability to recognise particular shapes and forms from a very young age, while our neurons and synapses are still plastic; this is an adaptation (in Darwinian terms an exaptation) of our innate abilities to extract patterns from our visual senses:
…one wonders just how many potential synaptic connections were lost to members of the civilized world when their ancestors abandoned a foraging lifestyle and settled in villages, towns, and cities. Anyone who has seen an African tracker scan a featureless plain and locate a distant pride of lions, invisible to everyone else in the car, will appreciate the impoverished nature of the synapses in his or her own visual cortex.4
Rather than impoverished, the same abilities are still developed but are now used differently to effortlessly discern the shapes and forms of symbols, and they are continuing to evolve and adapt under the pressure of increasingly complex graphical representations. Consider the bewilderment our hunter/gatherer would experience if displaced to the complex environment of a modern airport or train station, we are able to read effortlessly read the visual cues at a glance and so gain a knowledge of past and future events.
Historically, the power of ‘naming’, of being able to abstract and label a concept and so render it symbolic, has been a source of fascination. For instance, the epic of Gilgamesh (the first recorded story), tells of a complex character, simultaneously a wise leader and ruthless tyrant, part man and part god. In the story, Gilgamesh had mysterious hidden knowledge of the true names of things recovered from earlier times. It is through words that he secured his immortality by carving his story on stone.5
Modern fantasy writing is littered with references to those who gain power over others and the natural world via the knowledge of ‘true’ names or occult languages. The hermetic maxim of ‘as above so below’ is the essence of this ‘magical’ knowledge; if you find a way to abstract an aspect of the physical world to the ‘transcendent’ sphere of ideas, then manipulation of the idea gives control over the physical world.6
'Transient Taxonomies of Art' Bill Hart, 2008As with the cautionary tales of wizards who misuse their power, metaphors can be dangerous things - they can grant power, but at the same time blind us to other possibilities. We have become so fluent at reading that it seems as though the translation of image to symbol to concept is transparent, forgetting that, like the hunter on the savannah who can effortlessly read the terrain, we are looking at images. It is our training to read them as symbols; the associations built up with particular shapes and forms give them their specific meaning. This is nothing like the way a machine processes symbols.
We tend to forget that language is not static, it constantly evolves. The meanings of a word change with usage, new words are coined to describe a new experience, concept or object. Some become antiquated, obsolete or are made irrelevant by new insights. Some word/concepts and syntactical structures are so powerful that they can impede other ways of seeing and understanding. Notions of self, consciousness, and causality, for instance, deeply colour our thinking and dialogue about the nature of existence.7 Endless debates about mind/body dualities, objectivity/subjectivity and other apparent paradoxes are born from the limitations of the entrenched conceptual and perceptual framework, and the syntactical structure of language, rather than external fundamental truths.8
Visual images, as subjective representations, have the potential to be able to question the relevance of a worldview or conceptual framework in ways in which an analytical approach based in language cannot. A visual representation can give form to an intuition or a hunch; make explicit the tensions that exist in any state of being. Not forgetting, that some writers also have the ability to use words to paint pictures in the mind that ‘have the property of being in a special way “like something”, a something that again is very hard to capture in words.’9
References
1. Nicholas Humphrey, Seeing Red: A Study in Consciousness, Mind/Brain/Behavior Initiative (Cambridge, Mass.; London: Belknap Press of Harvard University Press, 2006), p112.
2. Steven Pinker. The Language Instinct : The New Science of Language and Mind (London: Penguin, 1995).
3. Barbara Maria Stafford, Visual Analogy: Consciousness as the Art of Connecting (Cambridge, Mass.: MIT Press, 1999), p8.
4. Daniel Lord Smail, On Deep History and the Brain (Berkeley: University of California Press, 2007), p137.
5. See Tablet 1, Maureen Gallery Kovacs, The Epic of Gilgamesh (Stanford, Calif.: Stanford University Press, 1989).
6. Translated by Dennis W. Hauck the full form is "That which is Below corresponds to that which is Above, and that which is Above corresponds to that which is Below, to accomplish the miracle of the One Thing." See Christopher R. Abel and William O. Hare, Hermes Trismegistus: An Investigation of the Origin of the Hermetic Writings (Sequim, Washington: Holmes Publishing Group LLC, 1997), p12.
7. In linguistic theory this is expressed as the Sapir – Whorf hypothesis; in its strong form the hypothesis implies that language determines what we can think. More generally accepted are weaker versions of the hypothesis based in relativity – ‘people who speak different languages perceive and think about the world quite differently’. See Daniel Chandler, The Act of Writing: A Media Theory Approach (Aberystwyth: Prifysgol Cymru, 1995), p14.
8. Quantum physicist, David Bohm, aware of the limitations that the emphasis on nouns in English places on our understanding of causality and process, developed a language called the rheomode that he forced his graduate students to use. See Chapter 2, “The rheomode - an experiment with language and thought” in David Bohm, Wholeness and the Implicate Order (London: Routledge, 1980), pp34-61.
9. Humphrey, Seeing Red: A Study in Consciousness, p112.
Bill Hart is an artist and lectures in Electronic Media at the Tasmanian School of Art in Hobart. Animations and further descriptions of his work can be found at www.billhart.id.au