Electronic Signs, Jay David Bolter

Visual Texts

Many writers have drawn analogy between the illuminated manuscript and the computer screen. Illuminated writing in which text is usually accompanied by a frame of visual information and decoration. Elaborate marginal glossing and intertextual patterning often made illuminated medieval works as beautiful as they were unreadable. The materiality of individual letters, or conversly, their sublimation into rythmic patterns mirrors the intermingling of image and text which characterises much multimedia. :

"On the screen, as in medieval parchment, verbal text and image interpenetrate to such a degree that the writer and reader can no longer say where the pictorial space ends and the verbal space begins"
Electronic Signs, Jay David Bolter

Illuminated manuscipts also share many elements with cartoons and comics. Their elaborately decorated initial letters often provided a gloss or an alternative viewpoint on the text. Often images almost entirely tell the story , with banners of dialogue and scrolls of explanation fluttering magically around hieractic figures. The comic replaces these with speech balloons and dramatic action, but similiarly the interplay of image and text generates a gestelt which is more than the sum of its parts..

George Legrady finds a close relationship between the conventions of comics and interactive multimedia:

Interactive, non-linear multimedia projects are based on the combining of images with texts and sounds to create narratives, or interactive situations. Close relatives to this medium include cinema and comics. Cinema is produces meaning through the sequencing of time based scenes. Comics also use sequence but instead of time and sound, there is the addition of texts and the use of multiple images on the page to create meaning.

• Sequential Art: Definition
  A story through the sequence of images and texts.
• icons
  Comics rely on simplified visual symbols, icons to carry meaning. Simple signs allow the viewer to imagine without effort.
• Abstraction
  Images are quickly recognized. Texts have to be learned and be decoded. They require more work.
• Closure
  Closure: Observing parts of things/situations but perceiving the whole. Closure is the means by which we can group information and make sense of things.
• Comic Narrative Sequence:*
  Closure and narrative flow in comics can be achieved a number of ways. These are six transitions from the most natural to the hardest in terms of recognizing closure.
• moment-to-moment
  Like frame sequences in films, requires little closure.
• action-to-action
  Ball comes to batter, batter hits ball.
• subject-to-subject
  Couple are ready to kiss, the phone rings
• scene-to-scene
  Scenes where we are transported across time or space require more reasoning
• aspect-to-aspect
  Where time goes by, transitions are like a wandering eye across aspects of a place, idea, mood.
• non-sequitur
  Sequences and juxtapositions which offer no logical relationship between them.
• Large panel-to-detail segments
  Small panels providing close-up detail are set next to a large image.
• Narrative reduction
  A narrative told in 50 panels can be reduced down to 4 to 6.
• Text only panels
  Where texts become iconic they allow for different types of interpretation
• panel shape creating a sense of time
  Long horizontal panels signify longer time.
• non-linear sequence
  We are used to reading from left to right but this can be played with.
• motion through static images
  Action be represented through repeated showing of an image.
• Visualizing emotions
  All icons can express moods, feelings, etc..
• visual metaphors
  Connotation - to imply rather than to describe, can also be done through simple icons (wavy lines to represent smell)
• balloons
  Balloons are containers of texts can also be visually altered to signify - example: icy looking balloon to represent someone being cool/distant.
• iconic representation
  Images are images but texts exist both as coded signs and visual elements.
• montage
  The combination of scenes can be related in a number of different ways:

• Word Specific
  Combinations where pictures illustrate but don't add much to the text
• Picture Specific
  Where words do little more than add a soundtrack
• Duo Specific
  Panels in which words and pictures send the same message
• Additive
  Words amplify or elaborate on the image or vice versa
• Parallel
  Where words and pictures follow very different courses without intersections
• Montage
  Where words are treated as integral parts of the picture
• Interdependent
  Where words and pictures go hand-in-hand to convey an idea

...all writing is a hybrid of art and technology. Pictographic and ideographic systems of writing are a more spectacular instance of the geometric nature of all written signs, and the calligraphic principle behind such systems demonstrates the interface between the pragmatics of communication and the artifice of visual beauty that guided the development of early scripts.

p. 47 Tofts 'Memory Trade'