Semiotic mode: Verbal Text
The semiotics of linguistic or verbal text has been dealt with fairly extensively. Barthes goes as far as to say that ‘every semiological system has its linguistic admixture’, and even the ‘meaning’ of a ‘visual substance’ is ‘confirmed by being duplicated in a linguistic message (Barthes, 1967: 10). However, in looking at the ideas of Kress and van Leeuwen (1996), and also of O’toole (1994), it is as reasonable, if not more so, to acknowledge a nonlinguistic manner with which meaning may be expressed, and interpreted. That will be dealt with in the following sections.
Nevertheless, in looking into the langue of verbal text, the lexicogrammar and the phonology constitute that supposedly universal context within which all users of the same language may draw resources for realizing their thoughts through the written word. However, as Kress and van Leeuwen mentioned, the context within which the individual’s parole is exercised is necessarily social. For spoken text, the occasion of the speech act delimits its semiotic potential. For written text, the literary mode and genre of the book or journal or newspaper does the same thing. Thus, there is little semiotic potential for a biased, personal, impassioned rant in the context of a public seminar or an academic journal, for the user who is aware of the rules and norms of that linguistic mode.
In this manner, the electronic writing space displays a greater semiotic potential with its nature of linkage and even ‘placelessness’ within cyberspace. Hypertext writers work in an ‘open-ended, perpetually unfinished textuality’ (Landow, 1997: 3), where genre and linguistic mode is page or site specific and where links may easily be made from other texts of other genres and other contexts in a purposeful manner to the local text. The semiotic potential of the electronic text, especially in terms of sign making, is thus increased because of the ‘galaxy of signifiers’ (in Barthes’ terms) (Barthes, cited in Landow, 1997:3) available in which the sheer variety of social and linguistic contexts blurs the distinctions and somewhat reduces the importance of context. Similarly, the reader of hypertext is not as contextually positioned as the audience of a talk, or the reader of a scientific journal. The result is not just a less limited semiotic potential, but also greater demands made on the user of the electronic writing space
Semiotic mode: Visual Text
Each semiotic mode has its own sets of interpretive frameworks suggested by theorists familiar with the characteristics of the mode, and for the visual mode, one of the most prominent concerns is with the ‘composition’ of a visual text. Composition is ‘the way in which the representational and interactive elements are made to relate to each other, the way they are integrated into a meaningful whole’ (Kress and van Leeuwen, 1996: 181).
In their book ‘Reading images: the grammar of visual design’ (1996), Kress and van Leeuwen suggest that the compositional elements of a picture/image are related through three interrelated systems: Information value, Salience and Framing (p.183).
To explain briefly, the information value of elements is embodied in their placement and position within the picture. Dissecting the space of a picture into left and right, top and bottom, center and margin, such information values as ‘given’ and ‘new’ (left and right), ‘ideal’ and ‘real’ (top and bottom) and ‘dominance’ (center) are assigned in the process of interpreting the logic and ‘meaning’ behind such a composition. The salience refers to the degree to which an element attracts the viewer’s attention and this is realized through such devices as foregrounding, relative size, contrast in tonal value, and so on. The presence or absence of a framing device disconnects or connects elements, suggesting the relationships between these elements and their wider significance to the image as a whole.
These techniques are applied not just to pictures and photographs, but also to the compositional aspect of such artifacts as newspapers and magazines, as demonstrated in examples of analyses of newspaper layouts by the authors. The significance of composition to the electronic writing space is fairly obvious; the webpage, appearing as a spatial collection of visual elements (whether lexia or headlines or pictures), lends itself to such a method of analysis and interpretation. As a brief example, the website for ‘Asiaone’ http://www.asiaone.com.sg/ may be analysed using this framework. The most obvious example would be the top/bottom dichotomy between the ‘ideal’ and the ‘real’. Here the title of the page sits at the top, as the ‘ideal’ or nominal identity of the webpage, while the actual ‘real’ contents appear below. This has relations to the relative ‘weight’ of each element, but that shall not be discussed at length.
The semiotic potential, or resource available for the maker of the webpage, thus takes the form of such an interpretation of compositional elements. Again, Kress and van Leeuwen stress that the langue here is a ‘meaning potential for “Western visual design”’ (1996: 8), again bringing to the fore the social delimitation of the resources available for meaning making.
On the printed page, such a process of analysis is aided by elements of fixity and boundary, providing a stable and environment in which the reader may begin making meanings from the signs he/she sees on the page without the fear of changes in the composition. On the electronic page, however, the perpetual quality of change and transience problematizes the interpretive process. Take for example, the option of scrolling. A page within the visible area of the browser screen seems, in the outset like a composition with top/bottom, left/right distinctions. However, a page made longer than the height of the browser window, or the computer screen, requires scrolling, and once scrolling is done, the initial ‘composition’ is disrupted, with bottom elements now appearing in the top space. Another significant example is the availability of frames. While the common use of a left margin frame (often as a navigation bar) adheres to the information valuation of ‘given’ and ‘new’, the navigational icons being the given, while the display frame on the right is the space where ‘new’ information appears, the differential scrolling, and changes in left frame content via a higher order navigation bar (perhaps at the top or bottom margins), adds considerable complexity to the compositional legibility and coherence to the webpage. Going beyond the singular, physical space of the printed page, frames bend and stretch the visible space shown on the computer screen and thus give an added dimension to the semiotic potential of the visual layout of an electronic text.