The paper discusses a practical model of the content of Virtual Environments (VEs) which provides a focus for the creative perceptual design of VEs and facilitates the further investigation of virtual environment theory. The model consists of two components: a characterisation of VE content in terms of its communicative potential, a simple structuring technique for organising content. The content model is illustrated by reference to a VRML model of the historic cliff lift at Saltburn by the Sea in the North East of England. Current work applying, validating, and incorporating the content model into a design methodology is then briefly overviewed. The paper is summarised and other directions for future research are listed.
If we take an inclusive definition of Virtual Reality (VR) to variously refer to desktop and high end VR, 3D computer games, Hybrid TV, and so, it seems that VR will constitute one of the principal communications media of the new century. We could predict with some confidence that such interactive 3D virtual systems will be to the twenty first century what the moving image was to the twentieth and will significantly change the way we view ourselves and the world around us. However, for a variety of reasons, our understanding of VR as a communications medium is not as well developed as the technologies of VR themselves. Thus our ability to construct effective, user centred virtual environments (VEs) is still very much reliant on individual knowledge coupled with prototyping and incremental development. The problem with such knowledge is that it is not generic and does not easily allow us to apply it to other applications areas particularly within the inclusive scope of VR we are taking.
In the field of computer games Church (1999) recognises just this problem when he calls for a set of "formal, abstract design tools" (FADTs) for the analysis, comparison and design computer games. In software engineering we would use a generic language such as UML to build abstract models of particular systems prior to implementation. UML has in fact been used in the design of VEs but was not intended to deal with the communicative and aesthetic qualities so essential to a medium such as VR.
In this paper a generic model of the content of VEs is introduced which will perform the kind of practical, generic role that Church calls for but in the wider context of VR in general. The content model for VEs functions something like the syntax and grammar of a natural language. It is based on the aesthetics of VR and focus on the fundamental communicative properties of VE content. It also provides a generic structuring mechanism that is used to relate communicative components into a coherent whole. The content model is illustrated with references to a virtual model of the historic, water balanced cliff lift at Saltburn by the Sea in the North East of England. The model is a virtual tourist site developed for the Saltburn Improvement Company (Fencott,1999a).
If VR is ever to be more than motor skills training or a generator of visceral thrills we will need to master the more subtle communicative qualities it surely possesses. How can we, for instance: create atmosphere, suspense and drama, trigger appropriate emotional responses, convey concepts and abstract ideas, facilitate the experiential creation of narrative.
Section 2 of this paper presents a brief overview of the diverse range of relevant material that has been brought to bear on the problem of constructing a practical content model of VEs. From this body of material a set of criteria, which a practical content model of VEs should satisfy, are laid out. In section 3 the Perceptual Opportunities (POs) model of the content of VR are discussed in the context of the VRML model of the cliff lift at Saltburn referred to above. In section 4 Perceptual Maps are introduced as a structuring mechanism for POs. In section 5 current work in validating the model is briefly reviewed along with suggestions for a design methodology for VEs are offered. Section 6 summarises the paper and suggests future research.
2. Locating the Content of VEs
Digital media, in general, have received and continue to receive a lot of attention from researchers in such diverse fields as media studies, sociology, psychology, memetics and so on as well as intense technological research into the embodying interface and its psychological and physiological effects. Despite this attention the actual nature of the content of Virtual Environments is rarely considered and even more rarely from the practical point of view of designing and building a VE to meet particular requirements and constraints. There are exceptions and this section briefly discusses, first of all, some of the available material on the aesthetics of VR. Secondly, a brief review of lessons from film theory is presented and finally a review of relevant research into VR mainly from a social science perspective. From this review a set of criteria can be identified which will ground the definition of the content of VEs that we are looking for.
Gombrich refers to Raphael's achievements in the compositions of his paintings as "[t]he perfect and harmonious composition of freely moving forms" (Gombrich,1977). One of the problems in VE design is that the visitors are freely moving and thus our encounters with the content components of a VE whether animate or inanimate, moving or static are also freely chosen. How therefore do we compose these elements appropriately let alone perfectly and harmoniously? Given the limitations on content imposed by the constraints of real time operation, how do we decide what content inclusions to make and the fidelity of these?
Before defining the content of VEs let us consider what the content of an older communications medium might be. What, for instance, would the content of novel be? Notice that we have not asked what the content of a book is for that would be to only consider the interface technology and not the communications medium itself which must consider both the medium as well as the interface technology.
Is the content of a novel the letters, words, sentences, paragraphs, pages, chapters or something else entirely? To a certain extent all of these could be considered content components but they are not all helpful when trying to write or analyse a particular novel. We want to identify a content model of the novel that helps us to understand novels and, more importantly, helps us to write novels. We need to find the point in the hierarchy of interrelationships between objects that make up a novel where we loose the uniqueness of the individual novel. Chapters, for instance, are too big as are paragraphs because such objects will almost certainly be unique to one particular novel. However, with the sentence we loose the uniqueness of the individual novel. This is because the same sentence could appear in many novels. For example the sentence: Sam got up and tried to leave, could be a sentence in hundreds of novels and even more if we could replace Sam with a generic variable to create a template for a sentence. Sentences are the largest objects that are too small to be unique to a given novel. However, although sentences are not unique to any one novel, the linear configuration of sentences will be exactly what constitutes the uniqueness of a particular novel. So content is not just about objects but about the form of configuration of threshold objects, such as sentences.
What then is the content of VEs? At the one extreme we have pixels and the mediated display while at the other end we have the scene graph nodes: polygons, texture files, sound files, proximity sensors and so on are both potential candidates. But none of these are the threshold objects we are looking for, as they are all not unique enough. We are closer when we consider objects such as tables, chairs, cars, people etc. which is the definition of content adopted by (Lombard and Ditton, 1998). However, objects in a VE will not always be in view or if in view may be too far away to be identifiable. They do appear to be threshold objects in the sense that tables and chairs etc. will be common to many VEs but their configuration will make the VE unique. Moreover, the same 3D model of a goblet, for instance, may be an exhibit in a virtual museum, an ornament in an architectural simulation, or a source of new powers in a real-time strategy game. The content model we are looking for must say important things about the communicative properties of such objects within the particular VE in which they are located.
The largest body of material available on designing content for VEs comes from the computer games world. Computer games are not only subject to the type of commercial pressures of film and television, for instance, but they are also without doubt the largest and most complex VEs built to date. In this context, Church calls for a set of "formal, abstract, design tools" (FADTs) that will not only guide the design of successful games but which will also enable designers to compare and contrast computer games from diverse genres (Church, 1999). Church's FADTs are perhaps better understood as an aesthetic characterisation of computer games and are:
Other computer games designers talk in a similar vein: of players needing to feel in control, of maintaining the emotional feel of a game and/or level, of providing suitable and timely rewards for effort, and of a perceivable gross structure which allows players to identify what is required of them at the beginning of a level, plan to achieve this and understand the significance of their achievement (Saltzman,1999).
From the field of media studies Murray (Murray,1996) identifies the following aesthetic characterisation of interactive media as:
Bringing these two views together we can characterise the aesthetics of general VR as:
Of course this is not the only characterisation of the aesthetics of VR. A number of artists have documented their aesthetics of VR but space does not permit us to pursue them further at this stage (Davies,2000; Moser,1996). The characterisation we have is a highly practical one and will suit our current purposes well.
On a more concrete level, Murray equates the structure of interactive media with the notion of the labyrinth and asserts that this structure works best when its complexity is somewhere between the 'single path maze' and the 'rhizome' or entangled web (Murray,1996). Aarseth has proposed the notion of cybertext to capture the class of texts, not just digital, which require the visitor to work to establish their own path(s) through the possibilities offered. He calls this class of text ergodic from the Greek words meaning work and path (Aarseth,1999). So we have a notion of a labyrinth which requires effort to explore. Equating the structure of VEs in general with the notion of a labyrinth of effort would seem useful but poses several questions. First of all, what are the actual components with which VE designers build such experiential labyrinthine structure? Second, how do VEs designers structure a VE so that the visitor follows an appropriate path and, moreover, accumulates an appropriate set of experiences so as to discover and remember the intended purpose of the VE?
Space syntax has much to teach us about the generic function of configurations of space and their effect on perception and behaviour. From this we have that the axis is of primary importance: 'The axis is fundamental because the experience of architecture is the experience of movement' - Hillier referring to Le Corbusier (Hillier,1996). Much of VE design is about the construction of axes of view and movement and this relates to the notion of labyrinth discussed above. Murray identifies the "choice point" as a major source of dramatic potential in interactive media (Murray,1996). We will incorporate both of these into the content model in the next section.
The nature of the content of films has been been the object of study of film theory over the course of the century and such concepts as montage and mise en scene can usefully be equated to VEs. In film the editor takes the general construction of scenes and clips established by the director and attempts to construct a perceptually transparent sequence that tells the story. In VEs the designer establishes mise en scene but has largely to delegate montage to the user/visitor. Eisenstein saw the content of VEs as the film clip while later researchers saw it as more subtle perceptual moments that arose out of the configuration of mise en scene and montage (Eisenstein,1968; Dudley,1976).
In film the use of mise en scene and montage attracts our attention to content of particular significance to the plot of the film. The configuration of such content allows users to hypothesis about what is going to happen next and what film in general is all about. A content theory for VEs must allow designers to structure content to attract visitors attention, allow them to hypothesise, form goals, and plan the achievement of gaols. From games design we have already pointed to strong evidence to show that visitors should be rewarded for such efforts. Rewards could be a variety of activities, new knowledge, access to previously available content etc. (Saltzman,1999;Ryan,1999).
From film theory we can also learn that objects and configurations of objects play a variety of connotative roles delivering meaning in addition to their denotative meaning as themselves (Monaco,1981). In film and VR a rose denotes itself - a rose is a rose is a rose ... - but it can connote other more abstract things to do with, for example, love, pleasure, sensuality and so on. We have already pointed out the potential connotative meaning of the same goblet in different VE contexts. If VEs are to achieve particular purposes beyond motor skills and the visceral then objects in them will have to operate connotatively as well as denotatively and the content model we are to develop should cope with this.
Cognitive engineering seeks to model the cognitive processes involved when users interact with systems in terms of identifying, planning for, and achieving goals (Johnson,1999). For instance, the resources model has been developed from theories of distributed cognition to help VE designers reason about the resources both internal and external to the user which users will require in order to identify goals, make plans and undertake interaction sequences appropriately. The resources model has been applied to the design of VEs (Smith et. al.,1999). The resources model views many previous theories of interaction as interaction styles which can themselves be represented and require appropriate cognitive resources, both internal and external to the user, in order to be able to be successfully utilised. Latterly, some attempt has been made to deploy the resources model as a design methodology for VEs [Smith et al,1999]. The resources model has proved very useful in understanding the importance of gaols and plans as central to the user interaction. Usability for VEs has become an active area of research in recent years due to the number of usability problems with VEs that have been identified (eg. Kaur,1998).
It would seem indisputable that human beings have the ability for not being mentally present in an environment in which they are observably physically present. This sense of presence, the feeling of being there, referred to as immersion in the games world, is at the heart of our experience of virtual reality. A sense of presence is not restricted to experiencing VEs but is also associated with story telling and reading, and of course film and television. Rather, presence in VEs is just perhaps the most recent example of the degree to which humans are naturally inclined to be taken out of themselves, enjoy loosing themselves in environments artificially created through communications media of all sorts. An intriguing thought experiment using an extreme example of this can be found in Zhai . The difference is that VEs go further than any other communications media in generating artificial stimuli for the perceptual systems to interpret rather than delegating the interpretation and creation to the mind itself as is the case with novels and story telling.
Lombard and Ditton define presence as the perceptual illusion of non-mediation (Lombard and Ditton,1997). This characterises presence as the state of mind of a visitor to a VE as not noticing or choosing not to notice that that which they are experiencing and interacting with is artificially generated. They document the evaluation of the embodying interface of a VE in terms of presence seen largely as the degree of fidelity of sensory immersion. Much of the research to date into presence is particularly concerned with the embodying interface as well as researches into the mental state of people who are present in VEs. Immersion is thus the degree to which the technology of the embodying interface mediates the stimuli to the senses. Slater has shown that high degrees of sensory immersion heighten the emotional involvement with a VE (Slater et al,1999).
However, as presence is a mental state it is therefore a direct result of perception rather than sensation. In other words, the mental constructions that people build from stimuli are more important than the stimuli themselves. It is the patterns that we, as VE constructors, build into the various cues that make up the available sensory bandwidth for a given VE that help or hinder perception and thus presence. These patterns are the result of what is built into the VE and the way the user behaves in response to them. The fidelity of the sensory input is obviously a contributing factor but by no means the most important. In the context of the working VE builder, being able to identify and make effective use of the causes of presence is more important than the nature of presence itself. This means that it is the effective consideration of the perceptual consequences of what we build into VEs that will give rise to the sense of presence that we are looking for. In this sense it is the content of VEs that has the greatest effect on the generation of presence. Thus, for our purposes, content is the object of perception.
Not all researchers have seen presence itself as the main focus of interest. Ellis states that the design of VEs should focus on the efficient communication of causal interaction and that presence is an epiphenomenon of secondary importance for design (Ellis,1996). A successful VE will change appropriately due to our actions within it and this is the primary cause of presence. This view would also seem to be asserting that effective content design is of primary importance and that Aaserth's requirement that effort is needed to find a path through a VE includes effort expended to interact with and understand it in general.
There is a further insight from psychological research, vital to the understanding of VE design, and that is the importance of the unconscious mind in allowing us to operate in the world (Spinney,1998) This is not the unconscious mind of Freud etc. but an important, perhaps the most important, part of our perceptual understanding of the everyday world and our belief in its dependability. It would seem that much of what we know about the world we know unconsciously and that this knowledge is largely unavailable to us consciously. Blackmore documents results which suggest that the conscious mind is actually an illusion generated by the unconscious mind (Blackmore,1999). Whether or not the latter is true, the central role the unconscious mind plays in allowing us to function in the world, any world, means that the content of VEs must in part be designed to satisfy both the conscious and unconscious minds.
Finally note on transformation. Transformation is important to many communications media. One of the great pleasures of novels is see the world through someone else’s eyes. To view the world through the eyes of another creature, machine or alien being. VR is ideally suited to this and much of the success of 3D computer games is due to the player being able to be the hero or villan in some great and dangerous advanture. In such games the player can not only play and alien but through the real time graphics actually see the world as the alien would see it. The author is certain than one of the successes of the classic Hubble Space Telescope Virtual Training Environment (VTE) [Loftin et al,1994] was that members of the ground based flight team could actually become an astronaut for a while and experience some of the drama and spectacle of a space walk. To my knowledge and despite the insightful research into the effect and effectiveness of the Hubble, the question; "Did you enjoy being an astronaugt for a change?" was never asked. Yet I suspect this was a major experience for the subjects.
To summarise the discussions in this section we can establish a set of criteria for a model of the content of VEs. For VEs, content:
In the next section we will use these criteria to constrain the content model for VEs that we are striving for and allow us to make an abstract characterisation of both labyrinth and content as they relate to the creative design of VEs.
Perceptual Opportunities (POs) are a content model for interactive media and VEs in particular and were initially developed to assist in the teaching of VE design (Fencott, 1999b). The model addresses the psychological qualities of a VE that seek to gain and hold the visitors attention through the human senses and perceptual system. The perceptual is about details that arise naturally from the spatial world and involve the visitor both consciously and unconsciously. The latter is very desirable because there is something very fundamental about unconscious involvement - accepting a place or activity without thought. P
The content model we will introduce in this section will make this structure explicit and thus allow us to use it as both a design tool and a model with which we can construct experiments into the nature of users interactions with VE of all genres. It is very often the case that the overall goal of a VE will not be a particular place or object, nor will the means of achieving that goal be explicit and concrete but will require the user to explore, formulate and solve problems, and generally progress through such activities towards completion of the VE - in the various forms that completion might take in this context. The keyword here is opportunity. The art of VE design is surely to provide users with carefully structured opportunities to allow them to explore, strategise, and generally feel some sense of control over what they are doing.
Also, in terms of VE design, we will be able to ask ourselves the kinds of questions a model should help us answer. For instance, why do we choose to model: Some objects but not others? Some details but not others? Some activities but not others? How do we select: what sounds we will use? What textures to use? What surfaces we wish to texture?
Content for VE is thus the appropriate configuration of a set of perceptual opportunities (POs) allowing the visitor to accumulate over time a set of experiences which maintain a sense of purposive presence. We will call such a configuration a perceptual map and discuss its structuring mechanisms after a detailed discussion of perceptual opportunities themselves. Creative design in VEs is thus concerned with attracting visitors' attention through patterns of mediated stimuli which will achieve purpose if the visitor perceives and responds to them as the designer intended. This is why the design of VEs is so difficult and why the model proposed is so useful.
The perceptual PO theory consists of a set of syntactic categories, see diagram above, which can be seen as attributes of any object that might conceivably be placed in a VE. These attributes specify the way in which the object is intended to function as part of the overall communications package of the VE. The syntactic categories into which perceptual opportunities can be characterised identify their role in achieving purpose and it is their planned interaction that gives us the overall structure we are looking for. A perceptual map is thus a loosely grammatical structuring of POs which seeks to ensure that users construct an appropriate temporal ordering over their attentions and activities within the VE. The figure below shows how the range of perceptual opportunities may be broken down into three principle forms, each of which will be defined and investigated in the sections that follow.
However, rather than simply discuss perceptual opportunities we will illustrate them with pictorial references to a desktop VE of the cliff lift and seafront at Saltburn by the Sea in the North East of England. This VE has served as both workshop and proving ground for the content model itself and both have developed in tandem.
Saltburn by the Sea is a small Victorian seaside resort which was purpose built in the last century it having been simply a few seashore cottages used by fishermen and smugglers. As can seen from the illustration, right, cliffs rise from the beach and to help the journey between beach and town (at the top of the cliff) a cliff lift was built in 1884. This consists of two small carriages running on an inclined tramway and connected by cables running round pulley wheels situated underneath the operator's hut at the top of the cliff. The motive power is water based. Each car has a tank underneath which can be filled with water. When some passengers are ready to travel water is run into the tank of the car at the top of the tramway until its weight is greater than the car at the bottom. Letting the brake off a little establishes this fact. As the car at the top of the tramway moves down under the weight of the water and the passengers so the car at the bottom is pulled up. When the short journey of 207 feet is completed the car at the bottom discharges the contents of its water tank ready to be pulled up on the next run.
Over the years the sea front at Saltburn has changed little apart from the pier being shortened as a result of a storm in 1974. The buildings at the land ward end of the pier, seen white with red geometric patterns in this photograph, have been enlarged several times and now obscure the ticket office for the cliff lift which is at the foot of the tramway. Today the cliff lift at Saltburn is one of the oldest of its kind in the world and still carries some 70,000 people a year between town and beach and back.
One of the principle reasons for choosing to model the cliff lift was that it was a potentially unbounded, outdoor environment which would therefore constitute a greater test of the effectiveness of the design criteria. This is so because the visitor has to be persuaded to remain in the area modelled rather than being constrained by walls and other physical constraints.
The principle requirement for the world was that it had to appear fairly realistic but must be focused on offering a working model of the cliff lift that can be operated by the visitor and that will also allow the visitor to ride up and down on it. A sequence of prototype VRML worlds were developed which, first of all, concentrated on building activity and interaction into the world. Despite the fact that an apparently realistic model of a real place was being built it soon becomes obvious that it was an abstract model and that a whole series of choices need to be confronted as a result of the necessary process of abstraction. In other words, the nature of perceptual reality as opposed to objective reality was being confronted. The VRML model of the cliff lift itself can be found at the Saltburn Improvement Company's web site (Fencott,1999a).
Sureties are mundane details that are somehow highly predictable - their attraction is their predictability. They arise directly from the architecture of the space. Concerned with the logic of the environment unconsciously accepted.
The following quote gives an insight from photography into the nature of sureties in VEs:
Hence the detail that interests me is not, or at least is not strictly, intentional, and probably must not be so, it occurs in the field of the photographed thing like a supplement that is at once inevitable and delightful. (Barthes,1984)
Sureties are thus concerned with: Vection, Ego Scale, Perceptual Noise, Distance, Limit, Self Image, Past, Physics
Sureties are often about unremarkable objects such as lampposts and street furniture which support of many of the above. This is true because sureties for distance, as people would normally recognise them, are largely absent in VEs. Distance or depth sureties are very difficult and encompass loss colour with distance, depth of focus, loss of fidelity, small objects disappear faster with distance than large ones, and all this decreases with decreases in display resolution. This is also true for for the scale of objects and one's own avatar. Space should not be static and sterile but dynamic and messy, perceptual noise, - we are used to the real world being like this so it helps if virtual ones are as well (VEs and mess/clutter don't however go naturally together). A useful aphorism is that in interacting with the real world we are trying to make sense of too much information but that in interacting with VEs we are trying to make sense of too little. The railings in the illustration below support vection, ego scale, distance cues, perceptual noise and so on.
Sound is an important spatial surety in reality and greatly supports presence (Whitlock et al,1996). It gives important information about the nature and scale of the space that we are currently experiencing, i.e. small, large, inside, outside, etc. We are very susceptible to reflected sound as sureties in this sort of way. We are not very good at locating objects accurately in 3D space based on the sounds emanating from them. The nature of sound in VEs means that sound can be used for atmosphere etc. but not as well for spatial and directional cues. This depends on the nature of the sound system itself being used, i.e.. stereo, surround sound.
Some examples drawn directly from the cliff lift are:
Sureties are closely allied to the fidelity and immersiveness of the embodying interface in that both seek to convince the visitor that the mediated environment is real.
The idea for surprises as perceptual opportunities came from the "appropriately designed infidelities" of Whitlock et al (1996) who used them for emphasis in virtual worlds and thus to precipitate conscious learning. In other words, non-mundane details that are not predictable but they do arise however surprisingly from the logic of the space consciously accepted. Surprises therefore are designed to deliver the purpose of the VE by allowing visitors to accumulate conscious experience from which narratives can be constructed after the visit.
Surprises can be:
And there are three basic types:
POs can be both sureties and surprises depending on the context in which they are offered - there is no mutual exclusivity between them. Some things will be more or less surprising than others, eg. limit sureties may sometimes be perceived as choice points, the animated cliff lift cars are attractors at a distance but retainers when the visitor takes a ride. See also the section on sureties and surprises working together.
Attractors are perceptual opportunities which seek to draw the attention of a visitor to areas of interest, retainers, that will deliver interaction sequences whose collective aim it to satisfy the high level objectives of the VE. Attractors are the means by which users are tempted into setting goals for themselves. It is thus important that major attractors are associated with perceptual opportunities, retainers, which reward users/visitors with things to do, remember, excite, puzzle, etc. and which will allow thus allow them to feel they have attained the goal they set themselves as a result of the attractor. To aid in these further perceptual opportunities, connectors, should be carefully designed to make any time/space connections between attractors and their retainer(s) attainable and interesting in themselves.
All attractors rely on peoples' natural curiosity and their prime purpose is to draw people into areas of conscious activity, called retainers, which are designed to deliver the main purpose of the VE. Attractors should stimulate visitors to identify goals and objectives with which the attractors retainer could be associated. Visitors should be able formulate plans to achieve the gaols they have given themselves using available connectors, axes and choice points. See the section on perceptual maps for more details on this.
Attractors may be characterised according to the reasons they draw attention to themselves:
In addition an attractors may exhibit a combination of characteristics. For instance a flashing light may be partially or wholly obscured behind an archway or other object and yet its effects may still be perceived. A sensational object such as a localised sound or a seabird above a beach may be also dynamically configured in that it 'reappears' in different places to indicate an affordance, help or information, for example.
Numerous examples of attractors can be found in the cliff lift:
Connectors are configurations of POs that help visitors achieve goals by supporting strategic thinking, making plans, semantic matching, and exploration. Connectors are thus the means by which visitors make connections, both mental and 'physical', between attractors which stimulate goal formation and retainers which allow visitors to achieve their goals and deliver objectives specific to the purpose of the VE. The actual objective of a retainer might well be hidden or not clear from the point of view of its attractor(s) but lower level goal formation should lead visitors into situations where objectives can be realised. Connectors are the means by which visitors are coaxed into following a particular course, choosing between possible courses, or changing course perhaps because they are approaching the limits of the VE. There are three types of connectors:
There are various techniques that can be used including direction signs; new routes appearing, degraded reality and so on. They can be closely allied to guide and limit sureties that seek to achieve similar objectives through unconscious perception. Thus an axis might lead a visitor into a position where an attractor becomes perceivable and follow this to a previously undiscovered retainer. Choice points are noted by Murray to be of great dramatic potential in interactive media (Murray). However, Bleszinski cautions that the consequences of making choices should be at least hinted at so that a perceptive visitor will be able deduce the consequences of their choices or at least understand them after the fact (Bleszinski,1999). Deflectors are unconscious counterpoints of choice points which offer a choice between high levels and low levels of detail to lead the visitor into a loaded choice. As the culture and cultural acceptance of VEs grows it may well be the case that all deflectors will be perceived as unconscious guide and limit sureties.
Examples of connectors in the cliff lift are:
Retainers are major patterns of interaction that seek to deliver specific objectives of the VE and collectively therefore its purpose. Retainers should be designed to deliver the memorable experiences of the VE as well as ensuring that visitors linger appropriately from time to time as they move around the world. In virtual tourism for instance the longer visitors linger overall the more likely they are to find the virtual experience memorable and perhaps retain the desire to actually visit the place the VE is modelling. Retainers come in two forms, local and peripatetic. They seek to keep visitors in a particular place in the VE, in the case of the local form, or provide visitors with interactions that they can access from wherever they are in the VE, as is the case with the peripatetic form. The local form may be dynamically configurable as in such games as Driver where retainers in the form of getting out of collisions can occur anywhere in the game space.
In the context of the cliff lift the principle retainers are:
We have now covered the basic types of surprises that we can consider as the syntactic components of a simple language for VEs. Perceptual mapping will now look at the grammar of this language.
Shocks are not perceptual opportunities normally built into VEs but arise as by-products of the design and construction process. They give rise to perceptions that jar, that aren't received as expected in the established context of the VE. They draw attention to the mediated nature of the environment and thus undermine presence. Shocks are thus perceptual bugs that need to be actively sought out and eliminated.
The static nature of the sea as currently modelled is also something of a shock but difficult to remedy within the constraints of a desktop VR system.
Surprises should work together in patterns to form possible temporal orders on retainers and thus the coherent set of purposive experiences that are intended to deliver the purpose of the world. Perceptual maps, as these patterns are called, are thus an abstract characterisation of the comprehensible labyrinth. Attractors should draw attention to sites of retainers and, if properly designed, lead visitors around the world in a meaningful way using connectors. Attractors may also themselves be retainers. Seen from a distance an animated object may act as an attractor but when experienced close up the object may be some sort of vehicle to ride in and control thus becoming a retainer. Patterns of attractors, connectors and retainers may be quite localised and in effect work as games. The oystercatcher avoiding visitors works in this fashion.
A perceptual map consists of the specification of the set of intended perceptual opportunities and their intended configurations. This will mean that when implemented in a VE they will allow a visitor to accumulate, over time, a set of experiences that maintain a sense of purposive presence. A perceptual map should therefore contain specifications of the range of sureties and surprises together with the perceptual interrelationships. We have already stated that sureties are the mundane details that seek to establish the believability of the world.
Sureties should be specified in terms of the particular certainties they seek to establish, ie. vection, ego scale, perceptual noise etc. Of course sureties can also function as surprises of various types depending on context and so may appear under both headings.
Since surprises are of three forms namely attractors, retainers and deflectors they will require different kinds of information to specify them. However, some general rules apply:
The simplest way of documenting an intended perceptual map is by way of a table with three columns which relate attractor/connector/retainer triples. Rows indicated the suggested relationships left to right and cells give brief descriptions. In the context of the cliff lift we will thus have:
Animated cars at distance
Operate cars from anywhere in world (outside cars)
Animated cars at distance
Pier and promenade railings, doorways of amusement arcade and partial views of ticket office etc. form axes to the car at the bottom of the cliff, alignment of pier and promenade
Ride up and down in the cliff lift
Unusual red and white pattern on amusement arcade, and partial views through entrance ways
Pier and promenade railings, alignment of pier and promenade to form axes towards amusement arcade
Interactive soundscape inside amusement arcade
Shore birds seen flying
A complex pattern of choice points - steps, turnings, gaps in railings etc. - linking axes. Not everything is straightforward.
Birds always fly away when you get too close, you can chase them up and down the beach.
Example of a game as a nested pattern of surprises.
Centre of cliff path railing obscured by cliff
Paths and railings along cliff top or promenade and steps at cliff bottom.
Deflectors suggest the route to follow.
Cliff path, with views of the pier and beach or cliff top etc.
Perceptual maps have much in common with the way painters arrange the composition of a work so as to catch the viewers attention and lead it around the canvas in a particular way. Although it is not possible to tell a story in a VE as in a film or TV program, there is never the less an important narrative element to VEs which needs to be designed for. This refers to the purposive accumulation of experience. This is more obvious in 3D games or Virtual Training Environments (VTEs) such as the classic Hubble Space Telescope VTE used for training the flight team (Loftin et al,1994). Because of resource limitations, objects should only be placed in a VE if they provide a clearly identified perceptual opportunity integrated into the VE's perceptual map. They will then support the purpose of the world.
We can now reconsider the content criteria we developed in section 2. The perceptual map is the means by which VE designers can directly address agency and narrative potential in their models. Each line in the perceptual map addresses agency by relating attractor and connector pairs, intention, to a retainer, perceivable consequence. Patterns of such attractor/connector/retainer grouping not only reinforce agency but also seek to offer narrative potential if they give rise to a set of coherent and memorable experiences that allow visitors to tell meaningful stories about them. Presence is not something that can be directly coded in to a VE but arises from both the skilful construction of the POs and the emotional involvement they engender in visitors. This latter is influcnced by visitors’ expectations of the VE. We have more control over transformation in that the POs offered to visitors’ can strongly suggest who or what they are intended to be. The perceptual map, and the connectors in particular, can be seen as an abstract realisation of the comprehensible labyrinth. Sometimes the labyrinth may seem far more concrete than in the cliff lift - Doom type games for example – but then there will be the abstract, perceptual labyrinth underpinning it. The break down of POs into sureties and surprises also seeks to address both conscious and unconscious mind and the denotative and connotative meaning potential of virtual content.
Sureties and surprises VEs work together much in the way jokes do:
The first two lines are unremarkable and mundane, sureties. The third line comes as a surprise but is plausible from the logic of the first two statements. Jokes seem to be all much like this - you set up an imagined and consistent, however fantastical, world and then give it a bizarre, implausible twist which must somehow be derivable from the former. Sureties and surprises in VEs work together, supporting each other and thus the virtuallity they inhabit by seeking to catch and retain the attention of the visitor and thus maintain presence and belief. If a perceptual map constitutes the labyrinth then sureties are the means by which it is grounded, virtually, in a believable world.
In the context of the cliff lift we have:
It is worth pointing out that many objects will provide a variety of perceptual opportunities as both surprises - quite possibly several depending on the context - and sureties.
As a practically oriented model POs are being validated and put to use in a number of ways.
From the outset POs were intended as a practical model of VR content to support VE development. One of the roles of a model is to test it against data generated in appropriate manners in order to see how the model correlates with actual experience. Perceptual opportunities arose out of the direct practise of VE creation and not from VEs viewed as the object of investigation although it has been informed by that. In a very real sense the mode of investigation was the series of trials and errors the author encountered in learning to build VEs and trying to make sense of why some apparently obvious content inclusions appeared to go largely unnoticed while others perhaps less obvious caught visitors attentions readily. The process of trial and error was made positive through a series of observations of and discussions with some 200 users of the VRML model of the cliff lift discussed in some detail above. POs arose out of a desire to generalise the mistakes and successes of the cliff lifts development process. For several years now the model has been used to teach students, undergraduates and graduates, the principles of VE design. In discussion with students it has become clear that the perceptual map of the cliff lift does indeed achieve it purpose but also that the goals and plans users construct are not as straightforward as the map would like to suggest. For instance, many students do indeed say that they planned to get to the cliff lift itself, as a result of the animated cars on entry, but that they put off achieving that goal until they had explored more and found out if there was anything else to do. Through such exploration they would establish a number of goals, prioritise them and then set about planning and achieving them. The results are the desired ones but the process is more complex than the perceptual map would suggest.
A further form of validation that is proving very useful is the application of POs to the analysis of existing VEs and to their comparative analysis. To date a number of VE, such as Char Davis's 'Osmose', the Hubble Telescope virtual training environment, and a number of games such as Sin, Driver, and Thief have been analysed and compared. The preliminary results of this work can be found in (Fencott,2000) and show that often apparently unrelated VEs exhibit structural similarities. In effect POs can indeed be used as the "formal, abstract design tools" to compare and contrast VEs that Church calls for (Church,1999).
A different but complementary form of validation is to conduct experiments to ascertain whether the general and particular predictions of a perceptual map for a given VE induce the expected behaviours from users. For each of the various forms of PO there are measures which can be experimentally investigated. This work is at an early stage but pilot experiments have been run. One early finding is that the power of attractors is easily offset by the effort required to realise goals associated with them. This work is currently proceeding and an experimental investigation to correlate actual user data from interacting with the cliff lift VE, making use of eye tracking technology, with the predictions of the perceptual map for the cliff lift is about to be run.
Perceptual opportunities are not a universal panacea for the problems of VE design but focus particularly on designing VEs to communicate effectively. POs along with a variety of other techniques, many of them discussed in section 2, can be brought together within a design methodology for VEs (Fencott,1999c). Such a design methodology can be used in a number of ways. In the shorter term it can be used to investigate the relationships between a wide variety of design and analysis techniques relevant to VE construction. In medium to longer term the methodology could serve as a practical support for practising VE designers to make best use of the may research efforts in the diverse fields of relevant to VR but without themselves having to be experts in all these fields.
This paper documents a practically oriented model of the content of a wide class of VEs. It serves to focus the creativity of designers on the communicative potential particular to VEs. The model is in essence an abstraction of the structuring concept of the labyrinth viewed as a map of interrelated perceptual opportunities. At present the model is stable but still in the course of development in terms of finer detail. It is within the perceptual map that the creative design process for VE construction takes place. The characterisation of VE content as perceptual opportunities has been illustrated with reference to a VRML model of a historic tourist site in the North East of England. Through the active use of perceptual mapping a highly interactive and presence provoking VE has been built that consists of just 685K total file size and which runs acceptably on a Pentium 233MMx PC with 32MB of RAM.
These ideas arose out of the practical activity of VE building and are the result of trial and error, playing with prototypes and so as well as the observation of and conversations with some 200 diverse users of the cliff lift VE. It would seem that as the embodying interfaces of virtual reality become established and the sensory bandwidth more widely covered we will need to switch our attention to the nature and content of virtual reality if we are to build effective VEs. If presence is indeed primarily the perceptual result of appropriate content then a deeper understanding of the way in which the human perceptual systems respond to VE generated content will be a major research area for the near future. The use of perceptual maps introduces an intuitive structure within which to proceed. The model, as presented here applies to presence, as discussed above, but work by Slater (1999) indicates that the same techniques, sureties and surprises etc., are also applicable to Co-Presence - the sense of being present with others.
Work is now afoot to use perceptual opportunities to undertake a detailed analysis of the tropes of VR so that we can gain a more rigorous understanding of the communicative relationship between VE creators and users. We can then undertake an investigation into the underlying principles by which we can create atmosphere, suspense and drama, trigger appropriate emotional responses, convey concepts and abstract ideas, and facilitate the experiential creation of narrative.
Aarseth, Espen J.(1997), "Cybertext: Perspectives on Ergodic Literature", John Hopkins University Press.
Blackmore, S. (1999), "The Meme Machine", Oxford University Press.
Blesninski, Cliff, in Saltzman, Marc,(ed), (1999), "Games Design: Secrets of the Sages", Macmillan.
Church, Doug.(1999), "Formal Abstract Design Tools", Games Developer Magazine, August.
Davis, Char. (2000), list of publications at http://www.immersence.com/publications.htm.
Dudley, Andrew, J.(1976), "The Major Film Theories: An Introduction", Oxford University Press.
Eisenstein, Serge,(1968), "The Film Sense", Faber.
Ellis, Stephen, R.(1996), "Presence of Mind: A Reaction to Thomas Sheridan’s "Further Musings on Psychophysics of Presence"", in Presence, Volume 5, No. 2.
Fencott, Clive,(1999a), "Virtual Saltburn Cliff Lift" for the Saltburn Improvement Company, http://www.saltburn-by-the-sea.com
Fencott, Clive,(1999b), "Content and Creativity in Virtual Environment Design", in proceedings of Virtual Systems and Multimedia ‘99, University of Abertay Dundee, Dundee, Scotland.
Fencott, Clive,(2000) "Comparative Content Analysis of Virtual Environments Using Perceptual Opportunities", in proceeding of conference on "Digital Content Creation", Bradford, England.
Fencott, Clive,(1999d),"Towards a Design Methodology for Virtual Environments", Workshop on User Friendly Design of Virtual Environments, York, England.
Hillier, Bill.,(1996), "Space is the Machine", Cambridge University Press.
Johnson, Chris,(1999), "Taking Fun Seriously: Using Cognitive Models to Reason about Interaction with Computer Games", http://www.dcs.gla.ac.uk/~johnson/papers/um99/games.htm.
Kaur, Kulwinder,(1998), "Designing Virtual Environments for Usability", PhD Thesis, City University, London.
Loftin, R. Bowen, and Kenney, Patrick, J.(1994), "The Use of Virtual Environments for Training the Hubble Space Telescope Flight Team", http://www.vetl.uk/edu/Hubble/virtel.html.
Lombard, Mathew, and Ditton, Theresa,(1997), "At the Heart of It All: The Concept of Telepresence", in Journal of Computer Mediated Communication, Volume 3, No 2, September 1997 http://jcmc.huji.ac.il/vol3/issue2/
Monaco, James,(1981), "How To Read A Film: The Art, Technology, Language, History and Theory of Film and Media", Oxford University Press.
Moser, Mary, Anne,(1996), "Immersed in Technology: Art and Virtual Environments", The MIT Press.
Murray, Janet, H.(1997), "Hamlet on the Holodeck: The Future of Narrative in Cyberspace", Free Press.
Ryan, T.(1999) "Beggining Level Design", http://www.gamasutra.com.
Saltzman, Marc,(ed), (1999), "Games Design: Secrets of the Sages", Macmillan.
Slater, Mel,(1999), "Co-Presence as an Amplifier of Emotion", 2nd International Workshop on Presence, University of Essex, http://www.essex.ac.uk/psychology/tapestries/.
Smith, Shamus,. Duke, David and Wright, Peter,(1999), "Using the Resources Model in Virtual Environment Design", Workshop on User Centred Design and Implementation of Virtual Environments, York, England.
Spinney, L.(1998), "I Had a Hunch ...", in New Scientist, 5th September.
Stephenson, Neal,(1993), "Snow Crash", Penguin Books.
UMLVersion 1.1 Summary, http://www.rational.com/uml/resources/documentation/summary/.
Whitelock, Denise,. Brna, Paul, and Holland, Simon,(1996), "What is the Value of Virtual Reality for Conceptual Learning? Towards a Theoretical Framework",
Clive Fencott is a senior lecturer at the University of Teesside where he researches and lectures in virtual reality theory and design, and it applications. He is deputy director of the Virtual Reality Applications Research Centre which is a multidisciplinary grouping consisting of psychologists, civil engineers, and designers, as well as specialists in VR and AI.