Semiotic mode: Aural Text
As may be apparent by now, the spatial, visual aspect of the electronic writing space has so far been dominant in the consideration of its semiotic potential. The eyes, more than the ears of the user, are engaged in this multi-modal text. However, the electronic text goes beyond the multimodality of printed text in its emancipation from the physical limitations of a physical artifact such as a book. It is thus able to tap the semiotic resource of aural text, which is temporal and sequential.
Again, the analysis of composition becomes useful for the web reader’s meaning making process, and in the case where an aural text is extant (whether as a background sound, or as a link to streaming media), the normal, spatial interplay of ‘left’ with ‘right’, ‘center’ with ‘margin’ is complicated with the concurrent, juxtaposition of a temporally governed text. When headline can interact with picture by their relative positions combined with their individual meanings, greater cognitive and affective demands are made on the reader trying to engage in both visual and aural texts, to think both spatially and temporally. And not just as discrete, merely coincidental juxtapositions, but as mutually affective and integrative signs.
What of musical text? While spoken language at least relates to written text, being realizations of language in different modes, music is further removed. The reader
Semiotic mode: Hypertext
After examining the semiotic modes possible within the electronic writing space, it is clear that the overarching semiotic potential of the electronic media is not merely the sum of the semiotic potentials of each mode. As shown by Kress and van Leeuwen, the analysis of multi-modal texts looks at ‘parts as interacting with and affecting one another’ (1996: 183). That is the case for a multi-modal text like the printed newspaper. However, the very nature of hypertext enriches its semiotic potential and places greater demands on its users. As mentioned on the section about verbal text, the linkages and web-like structure of the electronic space is one way in which the electronic mode has greater semiotic potential than its print counterpart. At the same time, the dynamic quality of composition in hypertext also places other demands on the user.
The focus now shall be on the notion of hypertext as ‘writerly’ text. Landow (1997) made the assertion that hypertext is the answer to Roland Barthes’ notion of the ‘writerly’ text, because it ‘fulfills “the goal of literary work [which] is to make the reader no longer a consumer, but a producer of the text”’(Landow, 1997: 5). The implications on the semiotic processes that take place, the processes of meaning making/transmission and meaning construction/reception, are profound.
Jean Jacques Nattiez, who is most well known for his work on the semiotics of music, writes of the ‘semiotic tripartition’ which considers three ‘objects’: the poietic and esthesic processes, which roughly translates into the processes of creation and construction, and the trace, the immanent embodiment of what is created (1990: 11-12). Thus the poietic process embodies the creation of a signifier, the making of the ‘trace’, while the esthesic process entails a reception of the ‘trace’ and a construction of meaning from this trace. The esthesic process is ‘heavily dependent upon the lived experience of the “receiver”’ (p.12), and this is related to the social dimension that Halliday and Kress and van Leeuwen stresses perpetually.
The reason why Nattiez’s suggestion is mentioned is because it shows how the ‘writerly’ text might change the traditional notion of a semiotic process, that of the expression of a meaning from the writer, in the form of a sign, which is received by the reader and interpreted according to his own means. Thus, if the reader is no longer a ‘consumer, but a producer’ of the text, then the party involved in the ‘esthesic’ process will also be involved in the ‘poietic’ process, changing the ‘trace’/sign/text, and then receiving the changed trace, influencing his/her own ‘esthesic’ process. This is a very raw and unrefined description, but the point is made that the semiotic potential is greatly expanded into other areas, to include other resources of sign making, even if in the process, demands are made to the reader/writer of electronic text.