Other Free Encyclopedias » Online Encyclopedia » Encyclopedia - Featured Articles » Contributed Topics from F-J


virtual real computer inhabitants

Nobuyoshi Terashima
Waseda University, Japan


On the Internet, a cyberspace is created where people communicate together usually by using textual messages. Therefore, they cannot see together in the cyberspace. Whenever they communicate, it is desirable for them to see together as if they were gathered at the same place. To achieve this, various kinds of concepts have been proposed such as a collaborative environment, tele-immersion, and telepresence (Sherman & Craig, 2003).

In this article, HyperReality (HR) is introduced. HR is a communication paradigm between the real and the virtual (Terashima, 2002; Terashima, 1995; & Terashima & Tiffin, 2002). The real means a real inhabitant such as a real human or a real animal. The virtual means a virtual inhabitant, a virtual human or a virtual animal.

HR provides a communication environment where inhabitants, real or virtual, those are at different locations, come, see, and do cooperative work together as if they were gathered at the same place. HR can be developed based on Virtual Reality (VR )and telecommunications technologies.


VR is a medium composed of interactive computer simulations that sense the viewer’s position and actions and replace or augment the feedback to one or more senses such as seeing, hearing, and/or touch, giving the feeling of being mentally immersed or present in the virtual space (Sherman and Craig, 2003). They can have a stereoscopic view of the object and its front view or side view according to their perspectives. They can touch and/or handle the virtual object by hand gesture (Burdea, 2003; Kelso, 2002; Stuart 2001).

Initially, computer-generated virtual realities were experienced by individuals at single sites. Then, sites were linked together so that several people could interact in the same virtual reality. The development of the Internet and broadband communications now allows people in different locations to come together in a computer-generated virtual space and to interact to carry out cooperative work.

This is collaborative virtual environment. As one of collaborative environments, the NICE project has been proposed and developed. In this system, children use avatars to collaborate in the NICE VR application, despite being at geographically different locations and using different styles of VR systems (Johnson, Roussos, Leigh, Vasilakis, Marnes & Moher, 1998). A combat simulation and VR game are applications of collaborative environment.

Tele-Immersion (National Tele-Immersion Initiative=NTII) will enable users at geographically distributed locations to collaborate in real time in a shared, simulated environment as if they were in the same physical room (Lanier,1998).

HR provides a communication means between real inhabitants and virtual inhabitants, as well as a communication means between human intelligence and artificial intelligence. In HR, communication paradigm for the real and the virtual is defined clearly. Namely, in HR, a HyperWorld (HW) and coaction fields (CFs) are introduced.

Augmented Reality (AR) is fundamentally about augmenting human perception by making it possible to sense information not normally detected by the human sensory system (Barfield & Caudell, 2001). A 3D virtual reality derived from cameras reading infrared or ultrasound images would be AR. A 3D image of a real person based on conventional camera imaging that also shows images of their liver or kidneys derived from an ultrasound scan is also a form of AR. HR can be seen as including AR in the sense that it can show the real world in ways that humans do not normally see it. In addition to this, HR provides a communication environment between the real and the virtual.


The concept of HR, like the concepts of nanotechnology, cloning and artificial intelligence, is in principle very simple. It is nothing more than the technological capability to intermix VRwith physical reality (PR) in a way that appears seamless and allows interaction. HR incorporates collaborative environment(Sherman,2003), but it also links collaborative environment with the real world in a way that seeks to be as seamless as possible. In HR, it is the real and virtual elements which interact and in doing so they change their position relative to each other. Moreover, the interaction of the real and virtual elements can involve intelligent behavior between the two and this can include the interaction of human and artificial intelligence. However, HR can be seen as including AR in the sense that it can show the real world in ways that humans do not normally see it.

HR is made possible by the fact that, using computers and telecommunications, 2D images from one place can be reproduced in 3D virtual reality at another place. The 3D images can then be part of a physically real setting in such a way that physically real things can interact synchronously with virtually real things. It allows people not present at an actual activity to observe and engage in the activity as though they were actually present. The technology will offer the experience of being in a place without having to physically go there. Real and virtual objects will be placed in the same “space” to create an environment called an HW. Here, virtual, real, and artificial inhabitants and virtual, real, and artificial objects and settings can come together from different locations via communication networks, in a common place of activity called a CF, where real and virtual inhabitants can work and interact together.

Communication in a CF will be by words and gestures and, sometimes, by touch and body actions. What holds a CF together is the domain knowledge which is available to participants to carry out a common task in the field. The construction of infrastructure systems based on this new concept means that people will find themselves living in a new kind of environment and experiencing the world in a new way.

HR is still hypothetical. Its existence in the full sense of the term is in the future. Today parts of it have a half-life in laboratories around the world. Experiments which demonstrate its technical feasibility depend upon high-end work stations and assume broad-band telecommunications. These are not yet everyday technologies. HR is based on the assumption that Moore’s law will continue to operate, that computers will get faster and more powerful and communication networks will provide megabandwidth.

The project that led to the concept of HR began with the idea of virtual space teleconferencing system. It was one of the themes of ATR (Advanced Telecommunications Research) in Kansai Science City. Likened to the Media Lab at MIT or the Santa Fe Institute, ATR has acquired international recognition as Japan’s premier research centre concerned with the telecommunication and computer underpinnings of an information society. The research lasted from 1986 to 1996 and successfully demonstrated that it was possible to sit down at a table and engage interactively with the telepresences of people who were not physically present. Their avatars looked like tailor’s dummies and moved jerkily. However, it was possible to recognise who they were and what they were doing and it was possible for real and virtual people to work together on tasks constructing a virtual Japanese portable shrine by manipulating its components (Terashima, 1994).

The technology involved comprised two large screens, two cameras, data gloves, and glasses. Virtual versions were made of the people, objects, and settings involved and these were downloaded to computers at different sites before the experiment’s start. Then it was only necessary to transmit movement information of positions and shapes of objects in addition to sound. As long as one was orientated toward the screen and close enough not to be aware of its edges, inter-relating with the avatars appeared seamless. Wearing a data glove, a viewer can handle a virtual object by hand gesture. Wearing special glasses, he/she can have a stereoscopic view of the object.

Most humans understand their surroundings primarily through their senses of sight, sound, and touch. Smell and even taste are sometimes critical too. As well as the visual components of physical and virtual reality, HR needs to include associated sound, touch, smell, and taste. The technical challenge of HR is to make physical and virtual reality appear to the full human sensory apparatus to intermix seamlessly. It is not dissimilar to, or disassociated from, the challenges that face nanotechnology at the molecular level, cloning at the human level and artificial intelligence at the level of human intelligence. Advanced forms of HR will be dependent on extreme miniaturisation of computers. HR involves cloning, except that the clones are made of bits of information. Finally, and as one of the most important aspects of HR, it provides a place for human and artificial intelligences to interact seamlessly. The virtual people and objects in HR are computer-generated and can be made intelligent by human operation or they can be activated by artificial intelligence.

HR makes it possible for the physically real inhabitants of one place to purposively coact with the inhabitants of remote locations as well as with other computer-generated artificial inhabitants or computer agents in an HW.

An HW is an advanced form of reality where real-world images are systematically integrated with 3D images derived from reality or created by computer graphics. The field of interaction of the real and virtual inhabitants of an HW is defined as a CF.

An example of HR is shown in Figure 1. In Figure 1, a virtual girl is showing her virtual balloon to a real girl in CFa. Two adults, one real and one virtual are discussing something in CFb which is a coaction field for interpreting between Japanese and English. They must be able to speak either Japanese or English. A real boy is playing ball with a virtual puppy in CFc. The boy and the puppy share the knowledge of how to play ball.


An HW is a seamless integration of a (physically) real world (RW) and a virtual world (VW). HW can, therefore, be defined as (RW, VW).

A real world consists of real natural features such as real buildings and real artifacts. It is whatever is atomically present in a setting and is described as (SE), that is, the scene exists. A virtual world consists of the following:

  • SCA (scene shot by camera): Natural features such as buildings and artifacts that can be shot with cameras (video and/or still), transmitted by telecommunications and displayed in VR.
  • SCV (scene recognised by computer vision): Natural features such as buildings, artifacts, facts, and inhabitants whose 3D images are already in a database are recognized by computer vision, transmitted by telecommunications and reproduced by computer graphics and displayed in VR.
  • SCG (scene generated by computer graphics): 3D Objects created by computer graphics, transmitted by telecommunications and displayed in VR. SCA and SCV refer to VR derived from referents in the real world whereas SCG refers to VR that is imaginary. A VW is, therefore, described as: (SCA, SCG, SCV). This is to focus on the visual aspect of a HW. In parallel, as in the real world, there are virtual auditory, haptic, and olfactory stimuli derived either from real world referents or generated by computer.

Coaction Field

A CF is defined in an HW. It provides a common site for objects and inhabitants derived from PR and VR and serves as a workplace or an activity area within which they interact. The CF provides the means of communication for its inhabitants to interact in such joint activities as designing cars or buildings or playing games. The means of communication include words, gestures, body orientation, and movement, and in due course will include touch. Sounds that provide feedback in performing tasks, such as a reassuring click as elements of a puzzle lock into place or as a bat hits a ball, will also be included. The behaviour of objects in a CF conforms to physical laws, biological laws or to laws invented by humans. For a particular kind of activity to take place between the real and virtual inhabitants of a CF, it is assumed that there is a domain of knowledge based on the purpose of the CF and that it is shared by the inhabitants.

Independent CFs can be merged to form a new CF, termed the outer CF. For this to happen, an exchange of domain knowledge must occur between the original CFs, termed the inner CFs. The inner CFs can revert to their original forms after interacting in an outer CF. So, for example, a CF for designers designing a car could merge with a CF for clients talking about a car which they would like to buy to form an outer CF that allowed designers to exchange information about car with clients. The CF for exchanging information between designers and clients would terminate and the outer CF would revert to the designers’ CF and clients’ CF.

A CF can therefore be defined as:

CF={field, inhabitants (n>1), means of communication, knowledge domain, laws, controls}

In this definition, a field is the locus of the interaction which is the purpose of the CF. This may be well defined and fixed as in the baseball field of a CF for playing baseball or the golf course of a CF for playing golf. Alternatively, it may be defined by the action as in a CF for two people walking and talking, where it would be opened by a greeting protocol and closed by a goodbye protocol and, without any marked boundary, would simply include the two people. Inhabitants of a CF are either real inhabitants or virtual inhabitants. A real inhabitant (RI) is a real human, animal, insect, or plant. A virtual inhabitant (VI) consists of the following:

  • ICA (inhabitant shot by camera): Real people, animals, insects or plants shot with cameras, (transmitted) and displayed in VR.
  • ICV (inhabitant recognised by computer vision): Real people, animals, insects, or plants recognised by computer vision, (transmitted), reproduced by computer graphics and displayed using VR.
  • ICG (inhabitant generated by computer graphics): An imaginary or generic life form created by computer graphics, (transmitted) and displayed in VR.

VI is described as: (ICA, ICG, ICV).

Again we can see that ICA and ICV are derived from referents in the real world, whereas an ICG is imaginary or generic. By generic, we mean some standardised, abstracted non-specific version of a concept, such as a man, a woman, or a tree. It is possible to modify VR derived from RW or mix it with VR derived from SCG. For example, it would be possible to take a person’s avatar which has been derived from their real appearance and make it slimmer, better-looking, and with hair that changes colour according to mood. Making an avatar that is a good likeness can take time. A quick way is to take a standard body and, as it were, paste on it a picture of a person’s face derived from a photo.

An ICG is an agent that is capable of acting intelligently and of communicating and solving problems. The intelligence can be that of a human referent or it can be an artificial intelligence based on automatic learning, knowledge base, language understanding, computer vision, genetic algorithm, agent, and image processing technologies. The implications are that a CF is where human and artificial inhabitants communicate and interact in pursuit of a joint task.

The means of communication relates to the way that CFs in the first place would have reflected light from the real world and projected light from the virtual world. This would permit communication by written words, gestures, and such visual codes as body orientation and actions. It would also have sound derived directly from the real world and from a speaker linked to a computer source which would allow communication by speech, music, and coded sounds. Sometimes it will be possible to include haptic and olfactory codes.

The knowledge domain relates to the fact that a CF is a purposive system. Its elements function in concert to achieve goals. To do this there must be a shared domain of knowledge. In a CF this resides within the computer-based system as well as within the participating inhabitants. A conventional game of tennis is a system whose boundaries are defined by the tennis court. The other elements of the system, such as balls and rackets, become purposively interactive only when there are players who know the object of the game and how to play it. Intelligence resides in the players. However, in a virtual game of tennis all the elements, including the court, the balls the racquets and the net, reside in a database. So too do the rules of tennis. A CF for HyperTennis combines the two. The players must know the game of tennis and so too must the computer-based version of the system. This brings us to the laws in a CF. These follow the laws of humans and the laws of nature. By the laws of nature are meant the laws of physics, biology, electronics, and chemistry. These are of course given in that part of a CF which pertains to the real world. They can also be applied to the intersecting virtual world, but this does not necessarily have to be the case. For example, moving objects may behave as they would in physical reality and change shape when they collide. Plants can grow, bloom, seed, and react to sunlight naturally. On the other hand, things can fall upwards in VR and plants can be programmed to grow in response to music. These latter are examples of laws devised by humans which could be applied to the virtual aspect of a CF.


The applications of HR would seem to involve almost every aspect of human life, justifying the idea of HR becoming an infrastructure technology. They range from providing home care and medical treatment for the elderly in ageing societies, to automobile design, global education and HyperClass (Rajasingham, 2002; Terashima & Tiffin, 1999, 2000; Terashima, Tiffin, Rajasingham, & Gooley, 2003; Tiffin, 2002; Tiffin & Rajasingham, 2003; Tiffin, Terashima, & Rajasingham, 2001), city planning (Terashima, Tiffin, Rajasingham, & Gooley, 2004),games and recreational activities and HyperTranslation (O’Hagan 2002).

A scene of HyperClass is shown in Figure 2. In this figure, three avatars are shown: one (center) is a teacher. It handles a part of Japanese virtual shrine. The other two are students. They are watching the operation.


HR waits in the wings. For HR to become the information infrastructure of the information society, we need a new generation of wearable personal computers with the processing power of today’s mainframe and universal telecommunications where bandwidth is no longer a concern. Such conditions should obtain sometime in 10 to 20 years. Now, a PC-based HR platform and screen-based HR are available.

In 10 years, a room based HR will be developed. In 20 years, universal HR will be accomplished. In this stage, they will wear intelligent data suits which provide a communication environment where they come, see, talk, and cooperate together as if they were at the same place.


Virtual reality is in its infancy. It is comparable to the state of radio transmission in the last year of the 19th century. It worked, but what exactly was it and how could it be used? The British saw radio as a means of contacting their navy by Morse code and so of holding their empire together. No one in 1899 foresaw its use first for the transmission of voices and music and then for television. Soon radio will be used for transmitting virtual reality and one of the modes of HR in the future will be based on broadband radio transmissions.

This chapter has tried to say what HR is in terms of how it functions and how it relates to other branches of VR. HR is still in the hands of the technicians and it is still in the laboratory for improvement after trials. But a new phase has just begun. HR is a medium and the artists have been invited in to see what they can make of it.

Hypersensitization [next] [back] Hypatia of Alexandria

User Comments

Your email address will be altered so spam harvesting bots can't read it easily.
Hide my email completely instead?

Cancel or