Content and Creativity in Virtual Environment Design

Abstract

Key Words

1 Introduction

A whole range of models of the content of VEs are possible depending on the purpose for which the theory is intended. The work presented here arose out of the direct practise of VE creation and not from VEs viewed as the object of investigation although it has been informed by the latter. In a very real sense the mode of investigation was the series of trials and errors the author encountered in learning to build VEs and trying to make sense of why some apparently obvious content inclusions appeared to go largely unnoticed while others perhaps less obvious caught visitors attentions readily. The process of trial and error was made positive through a series of observations of and discussions with some 200 users of a working VRML model of an historic inclined tramway at Saltburn by the Sea in the North East of England. The model itself was designed as a virtual tourist site and has since been purchased by the Saltburn Improvement Company Ltd to enhance their own Web site devoted to the promotion of the town [1]. The content model for VEs that is developed here will be illustrated with references to this VE.

In a sense we can capture the essence of the problem by quoting Gombrich, out of context and with profuse apologies, as the problem of VE design is "[t]he perfect and harmonious composition of freely moving forms" [2]. Here Gombrich is actually referring to Raphael's achievements in the compositions of his paintings. The problem in VE design is that we the visitors are freely moving and thus our encounters with the elements of the environment whether animate or inanimate, moving or static are freely chosen. How therefore do we compose these elements appropriately let alone perfectly and harmoniously?

2 Locating the Content of VEs

Murrray [3] equates the structure of interactive media with the notion of the labyrinth and asserts that this structure works best when its complexity is somewhere between the 'single path maze' and the 'rhizome' or entangled web. Aarseth [4] has proposed the notion of cybertext to capture the class of texts, not just digital, which require the visitor to work to establish their own path(s) through the possibilities offered. He calls this class of text ergodic from the Greek words meaning work and  path. So we have a notion of a labyrinth which requires effort to explore. Equating the structure of VEs in general with the notion of a labyrinth of effort would seem useful but poses the following question.

How do VEs designers structure a VE so that the visitor follows an appropriate path and, moreover, accumulates an appropriate set of experiences so as to discover and remember the intended purpose of the VE?

Space syntax has much to teach us about the generic function of space prior to perception and that the axis is of primary importance [5]. 'The axis is fundamental because the experience of architecture is the experience of movement' - Hillier referring to Le Corbusier [5]. Much of VE design is about the construction of axes of view and and this relates to the notion of labyrinth discussed above.

The labyrinthine form we will adopt is discussed in section 3 but we are still led to ask a further question.

What are the actual components, the content, with which VE designers build structure?

A starting point is the purpose of the VE and thus the genre it belongs to. For instance a virtual training environment (VTE), a 1st person 3D shoot-em-up, and a virtual tourist site will all adopt differing media conventions. The game for instance would have background music as an integral component whereas this could be a potential distraction in the VTE and also in the virtual tourist site unless street musicians, for instance, were introduced.

One class of such components are morphemes or story elements which have been identified in the bardic tradition and also in folk tales [3]. Film theory also has much to teach us in this respect. The nature of the content of films has been often been the object of study of film theory over the course of the century and such concepts as montage and mise en scene can usefully be equated to VEs. In film the editor takes the general construction of scenes and clips established by the director and attempts to construct a perceptually transparent sequence that tells the story. In VEs the designer establishes mise en scene but has largely to delegate montage to the user/visitor. Eisenstein [6] saw the content of VEs as the film clip while later researchers saw it as more subtle perceptual moments that arose out of the configuration of  mise en scene and montage [7]. We can turn to psychologically based research into aspects of VEs to give us further insight.

It would seem indisputable that human beings have the ability for not  being mentally present in an environment in which they are observably physically present. This  sense of presence, the feeling of being there,  is at the heart of our experience of virtual reality. A sense of presence is not restricted to experiencing VEs but is also associated with story telling and reading, and of course film and television. Rather, presence in VEs is just perhaps the most recent example of the degree to which humans are naturally inclined to be taken out of themselves, enjoy loosing themselves in environments artificially created through communications media of all sorts.  An intriguing thought experiment using an extreme example of  this can be found in Zhai [8]. The difference is that VEs go further than any other communications media in generating artificial stimuli for the perceptual systems to interpret rather than delegating the interpretation and creation to the mind itself as is the case with novels and story telling.

Lombard and Ditton's define presence as the perceptual illusion of non mediation [9]. This characterises presence as the state of mind of a visitor to a VE  as not noticing or choosing not to notice that that which they are experiencing and interacting with is artificially generated. They document the evaluation of the embodying interface of a VE in terms of presence seen largely as the degree of fidelity of sensory immersion. Much of the research to date into presence is particularly concerned with the embodying interface as well as researches into the mental state of people who are present in VEs. Immersion is thus the degree to which the technology of the embodying interface mediates the stimuli to the senses. Slater et al [10] has shown that high degrees of sensory immersion heighten the emotional involvement with a VE.

However, as presence is a mental state it is therefore a direct result of perception rather than sensation. In other words, the mental constructions that people build from stimuli are more important than the stimuli themselves. It is the patterns that we, as VE constructors, build into the various cues that make up the available sensory bandwidth for a given VE that help or hinder perception and thus presence. These patterns are the result of what is built into the VE and the way the user behaves in response to them. The fidelity of the sensory input is obviously a contributing factor but by no means the most important. In the context of the working VE builder, being able to identify and make effective use of the causes of presence is more important than the nature of presence itself. This means that it is the effective consideration of the perceptual consequences of  what we build into VEs that will give rise to the sense of presence that we are looking for. In this sense it is the content of VEs that has the greatest effect on the generation of presence. Thus, for our purposes, content is the object of perception.

Not all researchers have seen presence itself as the main focus of interest. Ellis [11] states that the design of VEs should focus on the efficient communication of causal interaction and that presence is an epiphenomenon of secondary importance for design. A successful VE will change appropriately due to our actions within it and this is the primary cause of presence. This view would also seem to be asserting that effective content design is of primary importance and that Aaserth's requirement that effort is needed to find a path through a VE includes effort expended to interact with and understand it in general.

There is a further insight from psychological research, vital to the understanding of VE design, and that is the importance of the unconscious mind in allowing us to operate in the world [12]  This is not the unconscious mind of  Freud etc. but an important, perhaps the most important, part of our perceptual understanding of the everyday world and our belief in its dependability. It would seem that much of what we know about the world we know unconsciously and that this knowledge is largely unavailable to us consciously. Blackmore [13] documents results which suggest that the conscious mind is actually an illusion generated by the unconscious mind. Whether or not the latter is true, the central role the unconscious mind plays in allowing us to function in the world, any world, means that the content of VEs must in part be designed to satisfy both the conscious and unconscious minds.

To summarise the discussions in this section we can establish a set of criteria for a model of the content of VEs. For VEs, content:

• is the object of perception.
• should effectively communicate causal interaction.
• should take on the structure of the comprehensible labyrinth
• should appeal to the unconscious as well as the conscious mind.

3 Perceptual Opportunities

The content of VEs is defined as an organised set of perceptual opportunities allowing the visitor to accumulate over time a set of experiences which maintain a sense of purposive presence. We will call this structure a perceptual map and discuss this after a detailed discussion of perceptual opportunities themselves. Creative design in VEs is thus concerned with attracting visitors attention through patterns of mediated stimuli which will achieve purpose if the visitor perceives and responds to them as the designer intended. This is why the design of VEs is so difficult and why the model proposed is so useful.

Perceptual opportunities can be characterised according to their role in achieving purpose and it is their planned interaction that gives us the overall structure we are looking for. The figure below shows how the range of perceptual opportunities may be broken down into three principle forms, each of which will be discussed in the sections that follow.

3.1 The Cliff Lift at Saltburn

Over the years the sea front at Saltburn has changed little apart from the pier being shortened as a result of a storm in 1974. The buildings at the land ward end of the pier, seen white with red geometric patterns in this photograph, have been enlarged several times and now obscure the ticket office for the cliff lift which is at the foot of the tramway. Today the cliff lift at Saltburn is one of the oldest of its kind in the world and still carries some 70,000 people a year between town and beach and back.

One of the principle reasons for choosing to model the cliff lift was that it was a potentially unbounded, outdoor environment which would therefore constitute a greater test of the effectiveness of the design criteria. This is so because the visitor has to be persuaded to remain in the area modelled rather than being constrained by walls and other physical constraints.

The principle requirement for the world was that it had to appear fairly realistic but must be focused on offering a working model of the cliff lift that can be operated by the visitor and that will also allow the visitor to ride up and down on it. A sequence of prototype VRML worlds  were developed which, first of all, concentrated on building activity and interaction into the world. Despite the fact that an apparently realistic model of a real place was being built it soon becomes obvious that it was an abstract model and that a whole series of choices need to be confronted as a result of the necessary process of abstraction. In other words, the nature of perceptual reality as opposed to objective reality was being confronted. The importance of prototyping in VE development and the need for a focused and concise specification upon which prototypes can be incrementally developed to address important issues such as presence first and geometric detail later became clear. The VRML model of the cliff lift itself can be found at the Saltburn Improvement Company's web site from early June [1].

3.2 Sureties

The following quote gives an insight from photography into the nature of sureties in VEs:

Hence the detail that interests me is not, or at least is not strictly, intentional, and probably must not be so, it occurs in the field of the photographed thing like a supplement that is at once inevitable and delightful. Barthes [14]

• Vection
• Ego Scale
• Perceptual Noise
• Distance
• limits

• sound of the cars moving over the metal tracks to reinforce travelling motion
• railings around the pier, the promenade and cliff tops to indicate avatar scale, vection and to suggest where to go etc.
• It is worth noting the fact that none of the railings modelled bear more than a passing resemblance to those actually in place in Saltburn. Comparing the right to the photograph above should convince the reader of that. The actual and the virtual do not match. The important point is that there should be railings where railings are expected. the same is true for the railings on the pier which are of a different style in reality but the same as those below in the VE.
• similarly the actual structure of the pier supports are quite complex but only the outer angled supports modelled in the VE.
• eves on the amusement arcade and ticket office etc. which as the next illustrations show do not resemble each other yet seem to have passed the scrutiny of visitors to date. Texture files have been scanned at very low  resolution but this does not seem to have raised comments among the students who visited the site.

• eves, railings, pier supports, etc. all add perceptual noise in addition to their primary design justifications.

3.3 Surprises

Surprises can be:

• implausible but beneficial
• totally plausible but unexpected

• attractors
• connectors
• retainers

3.3.1 Attractors

Examples of attractors in the cliff lift are::

• The cars on the inclined tramway automatically undertake a journey 10 seconds after the world loads
• As a visitor moves down the pier towards the amusement arcade for the first time a shore bird, an oystercatcher, flies out from under the pier and lands on the beach
• Open doorways into the amusement arcade and the ticket office.
• Partial views of the ticket office from through the open doors of the amusement arcade.
• The little cabin, brake mans hut, and more railings partially visible at the top of the tramway.

3.3.2 Connectors

• Axes
• Deflectors

Examples of connectors in the cliff lift are:

• as the visitor travels away from the cliff lift along the promenade or cliff top degraded reality is used to indicate they are approaching the limit. In this case the uprights on the railings disappear along with the general level of detail (Deflector)
• railing along the pier provide sight lines to guide the visitor into the amusement arcade (Axis).
• paths from the cliff top down to the promenade seek to deflect visitors from getting too far away form the main focus of interest. (Deflector).

3.3.4 Retainers

In the context of the cliff lift the principle retainers are:

• Being able to control the cliff lift from anywhere in the VE using the peripatetic controls (green, yellow and red buttons) at the bottom of the screen (unexpected)
• Being able to ride on the cliff lift using the above controls (expected)
• The inside of the amusement arcade is empty of objects but a series of sound samples from the actual arcade fade in and out of hearing as a visitor moves around inside the building. An example of perceptual metonymy. (unexpected)
• The oystercatcher on the beach always flies away to a new spot on the beach if the visitor approaches too close. (expected)

3.4 Perceptual Maps

In the context of the cliff lift we have:
 

Attractor	Connector	Retainer
Animated cars at distance	None necessary	Operate cars from anywhere in world (outside cars) Peripatetic
Animated cars at distance	Pier and promenade railings, doorways of amusement arcade and partial views of ticket office etc. form axes to the car at the bottom of the cliff, alignment of pier and promenade	Ride up and down in the cliff lift Local
Unusual red and white pattern on amusement arcade, and partial views through entrance ways	Pier and promenade railings, alignment of pier and promenade form axes towards amusement arcade	Interactive soundscape inside amusement arcade Local
Shore bird seen flying	No straightforward axes, a complex pattern of deflectors - steps, turnings, gaps in railings etc. - not everything is straightforward In keeping with the retainer to follow	The bird always flies away when you get too close, you can chase it up and down the beach Local Example of a game as nested pattern of surprises

Perceptual maps have much in common with the way painters arrange the composition of a work so as to catch the viewers attention and lead it around the canvas in a particular way. Although it is not possible to tell a story in a VE as in a film or TV program, there is never the less an important narrative element to VEs which needs to be designed for. This refers to the purposive accumulation of experience. This is more obvious in 3D games or Virtual Training Environments (VTEs) such as the classic Hubble Space Telescope VTE used for training the flight team [16]. Because of resource limitations, objects should only be placed in a VE if they provide a clearly identified perceptual opportunity integrated into the VE's perceptual map. They will then support the purpose of the world.

3.4.1 Sureties and Surprises Working Together

• My dog has no nose!
• How does he smell?
• Terrible!

In the context of the cliff lift we have:

• the empty interior of the amusement arcade filled with appropriate sounds but no objects signals the use of the building but also that this is not central to the purpose of the world
• the fact that sounds, controls, animation, and vection come together in the cliff lift cars themselves establishes that they are the focus of the VE. Particularly in relation to the partially modelled amusement arcade.
• a similar thing happens with the gift shop, next door to the ticket office, whose fascia is modelled by a bit map and cannot be entered as opposed to the ticket office which is similarly textured but has real doorways and not images of them.

3.5 Shocks

In earlier versions of the cliff lift there were a variety of shocks such as the sea not being big enough so that visitors could see where it ended. The static nature of the sea as currently modelled is also something of a shock but difficult to remedy within the constraints of a desktop VR system.

3.6 Conclusions

These ideas arose out of the practical activity of VE building and are the result of trial and error, playing with prototypes and so as well as the observation of and conversations with some 200 diverse users of the cliff lift VE. It would seem that as the embodying interfaces of virtual reality become established and the sensory bandwidth more widely covered we will need to switch our attention to the nature and content of virtual reality if we are to build effective VEs. If presence is indeed primarily the perceptual result of appropriate content then a deeper understanding of the the way in which the human perceptual systems respond to VE generated content will be a major research area for the near future. The use of perceptual maps introduces an intuitive structure within which to proceed. The model currently applies to presence, as discussed above, but work by Slater [10] indicates that the same techniques, sureties and surprises etc., are also applicable to Co-Presence - the sense of being present with others.

Work is currently proceeding in a number of directions. First of all pilot experiments are being conducted, in conjunction with the School of Social Science at the University of Teesside, to investigate the way interrelated patterns of surprises are perceived and responded to by users. Secondly, the content model is being used to analyse a number of VEs, e.g.. a VTE, several computer games, and so on, to see if the model is as general as believed. Out of this the relationship between content and genre can also be considered. Finally, the content model is being used to develop a design methodology for VEs with a view to integrating the creative design process particular to VEs with technical, resource and project management issues.

References