Virtual Reality: Principles and Applications

Virtual Reality: Principl

Computer Science [cs]/Human-Computer Inte

12/27/2024

Frédéric Mérienne
[company name]
Table of Contents
Abstract............................................................................................................................................................3
INTRODUCTION................................................................................................................................................3
Principle of Virtual Reality................................................................................................................................3
Virtual Reality Issues.........................................................................................................................................6
VIRTUAL REALITY TECHNOLOGIES....................................................................................................................6
Sensory Interfaces............................................................................................................................................6
Motion Tracking...........................................................................................................................................6
Visualization Devices....................................................................................................................................7
Haptic Feedback...........................................................................................................................................8
Motion..........................................................................................................................................................8
3D Representation............................................................................................................................................8
Data Management........................................................................................................................................8
Computer Graphics.......................................................................................................................................9
User Interaction Techniques...........................................................................................................................10
Navigation..................................................................................................................................................10
Manipulation..............................................................................................................................................10
Tactile Interfaces........................................................................................................................................10
Model of Perception.......................................................................................................................................11
EVALUATION OF VIRTUAL IMMERSION..........................................................................................................11
INTERACTION IN VIRTUAL ENVIRONMENT.....................................................................................................11
Multisensory Interaction................................................................................................................................11
NATURAL USER INTERACTION........................................................................................................................12
Exploring the Virtual World............................................................................................................................12
Driving............................................................................................................................................................12
VIRTUAL PROTOTYPING..................................................................................................................................13
CONCLUSION..................................................................................................................................................14
REFERENCES...................................................................................................................................................14

Page 1 of 17
 Virtual Reality: Principles and Applications
Frédéric Mérienne

To cite this version:

Frédéric Mérienne. Virtual Reality: Principles and Applications. Encyclopedia of Computer Science and
Technology, Taylor and Francis, pp.1-11, 2017, 10.1081/E-ECST2-140000194. hal-01728062

HAL Id: hal-01728062 [Link]

01728062v1
Submitted on 9 Mar 2018
HAL is a multi-disciplinary open access archive recherche français ou étrangers, des laboratoires
for the deposit and dissemination of scientific publics ou privés.
research documents, whether they are published or
not. The documents may come from teaching and
research institutions in France or abroad, or from
public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de

Page 2 of 17
Virtual Reality: Principles
computer interaction enable to make the
link between the user and a virtual
environment in capturing the user’s
and Applications motion, acting on his senses as well as
computing the virtual experience in
Fre´de´ric Merienne real-time. The immersion in virtual
Le2i, Arts et Metiers, France environment is evaluated through the
user’s perception and reaction. Virtual
reality is used in a large variety of
application domains which need
multisensory interaction and navigation
Abstract
facilities. Virtual prototyping is also
Virtual reality aims at immersing a user
used in the industry to improve design
in a virtual environment. Dedicated
process.
virtual reality technologies of human–
INTRODUCTION virtual environment—multisensory interaction

V
and natural user interaction. The fifth part deals
irtual reality is widely used in with the navigation process in virtual
different application domains environment. Two main cases of use are
benefiting from the development of exposed—exploring the virtual world by
technologies coming from video game (as the miming walking, flying, or teleporting ways,
computer does). Virtual reality consists in and driving. A specific application for industry
providing the possibility to interact with a purpose is then discussed in the sixth part.
virtual environment. The human is in the center Virtual prototyping aims at using digital
of the application which needs dedicated representation in the design process of the
technologies for interaction. Thus, knowledge object. The principle and main issues of virtual
of technologies and human factors are required prototyping are exposed.
to build an efficient virtual reality application. This entry is written to give an overview of
This entry describes virtual reality principle virtual reality. Some general aspects are presented.
and some key features existing in several The reader who wants to explore this topic in depth
applications. The first part shows how can find lots of rich information in several books.
interactions occur in virtual environment.
Principle of virtual reality involves knowledge
of technologies and human factors because INTERACTION WITH THE VIRTUAL WORLD
multisensory feedback interfaces need to be
fitted with human senses as virtual immersion Principle of Virtual Reality
is evaluated on the human. The second part
presents virtual reality technologies. Main Virtual reality consists of software and hardware
sensory interfaces, which are constituted by technologies which interact with the user to
hardware systems, are presented regarding the immerse him in a virtual environment. The
human sense they address. 3D representation of principle is to create a relationship between the
the virtual environment is then discussed user and a virtual environment. For that, software
regarding the virtual data management and the technologies (computer graphics, real-time
graphics process. User interaction techniques computing) as well as hardware technologies
are performed in close link with sensory (human—computer interfaces) are required. Figure
interface and are designed as function of the 1 provides an overview of the relationship between
requirement of the application. The third part the user and a virtual environment. As described in
deals with perception of virtual immersion. The Fig. 1, movements of the user are captured in real
concept of presence is introduced to evaluate time in order to animate his avatar (virtual user),
virtual immersion from the user point of view. who interacts with the virtual environment. The
Model and measurement of presence are effects of interaction between the avatar and the
discussed. Virtual reality is applied in a wide virtual environment are
range of applications which require restitutedtotheusersensesthroughsensoryfeedbackte
specifications in terms of interaction in virtual chnologies. This interaction loop has to be
environment. The fourth part presents two performed in the delay of the refresh rate of the
important characteristics of interaction in

Page 3 of 17
 Sensory interfaces

Senses Image Sensory feedbacks Interaction

sound
haptic Avatar
... Virtual
excited human User
sense visualization devices, sound environment and a virtual mock-up
(commonly named as real-time feedback, tactile and force representation of the subject. A
process). feedback, as well as motion real-time interaction between
Then, virtual objects are
Motion platform,
Motion for example. Animation
capture the user and the virtual
perceived through technologies Interaction in virtual environment requires to capture
acting on human senses. These environment is realized through motion or intention of the user
technologies are constituted of a relation between virtual in real time in order to
Fig. 1 Relationship between the user and a virtual Fingers
environment. trackin
g
Stereoscopic
refresh the avatar position and motion in the virtual glasses
environment according to the user’s position and
motion. In that way, a coherence between physical Haptic
rendering
world and virtual world is realized in real-time.
(vibration)
The user’s perception in virtual environment is
evaluated through the concept of sense of presence
which depends on the way the user can explore the
virtual environment, the possibilities he has to
interact with it, and the multisensory feedbacks Fully
provided by the technologies. Sense of presence is immers
strongly dependent on the engagement of the body ive
in the virtual immersion process. Thus, virtual visuali
zation
reality tends to fully immerse the user in the virtual display
world as illustrated in Fig. 2. Multisensory
Feedback Technologies
Fig. 2 Full immersion of the user in the virtual
Virtual immersion is enhanced by multisensory environment.
feedback acting on the user’s senses. The senses and virtual reality technologies is
knowledge of human senses characteristics enables optimized when the main characteristics of
the design and implementation of virtual reality human senses are covered by technologies
technologies. Relationship between human acting on it. Table 1 summarizes the main
characteristics of human senses and its
corresponding technologies which have to be
considered for virtual reality applications.
Vision is one of the most important human
sense to perceive the environment. Human eye
is constituted of a lens (cornea and crystalline
lens), a diaphragm (iris), and a light sensor
(retina). Crystalline lens is warped by a
rotational muscle for adapting the vision to the
distance of the observed object. This warping
enables the accommodation of the eye in
relation to the distance of the object in order to
produce a distinct image of the observed object.
Due to the optical system (lens), the image
produced at the retina level is not distinct for
the whole scene but just around the area of the
observed object. This area is called the depth of
field from where all objects are distinct in the
Head image. Images of the objects situated outside
tracki this area are blurred. The retina is composed of
ng three different layers. At the light sensitive
layer, retina has billions of photoreceptors
which transform the luminous energy into

Page 4 of 17
electrical signal. A second retina layer (the evaluation on population lead to typical
inner retina) transforms electrical signal into characteristics of the average human eye. Thus, the
different combinations for a first image luminous efficiency of average human eye varies
processing operation (neighborhood effects by in the spectrum from 400 nm to 700 nm with a
bipolar, amacrine, and horizontal cells). A third highest value at 500 nm in low-light conditions
retina layer compresses the image information (due to rod) and at 550 nm in high-light condition
(on a wave train by ganglion cells) for a (due to cone). Average human eye is then sensitive
transmission to the brain. Two types of to a very large range of luminance (from 0.0001
photoreceptors exist in the retina—rod and cd/m² to 10,000,000 cd/m²) from star light to sun
cone. Rods are highly sensitive to the light and light. The field of view is approximately 180° on
give the capability to perceive at very low light the horizontal plane and 140° on the vertical plane.
conditions (moon light). Cones are sensitive to The perception of details in an image can be
sun light and are mainly disposed in the central analyzed as the minimal angle under which the eye
visual axis (foveal axis). Three different types can distinguish two points. This angle defines the
of cones exist depending on the pigment used to capability of eye resolution or visual acuity. Visual
transform light information into electrical acuity depends on several factors such as the
signal. Each of the three pigments are sensitive luminance, eccentricity, contrast, and motion of the
to different wavelengths, enabling the observed object. Under the best luminous
perception of color. Luminous efficiency of rod condition, the visual acuity on the foveal axis is
is approximately one hundred times higher than close to an angle of 1 arcmin. This angle
the luminous efficiency of cone. Because rods corresponds to 300 dots per inch for an image
and cones are not sensitive to the same level of visualized at 30 cm or 45 dots per inch for an
luminance, vision image visualized at 2 m.
Table 1 Relationships between human senses, main characteristics, and virtual reality technologies.
Human sense Main characteristics Virtual reality technologies
Vision Sensitive between 400 nm and 800 nm Dynamic screen
Huge range of brightness (from 0.0001 cd/m² to 10,000,000 cd/m²) Resolution capabilities
High resolution Large FOV screen
High-contrast sensitivity Eye tracking
60 Hz per eye for VR (head motion) Head-mounted display
Rotation speed up to 800°/s
180°FOV in horizontal plane
140°FOV in vertical plane
Stereovision Stereoscopic rendering area (Panum zone) Right/left channels splitting system
Visual accommodation and convergence relationship Head-tracking system
Sound High dynamic Ears tracking
Frequency range from 20 Hz to 20,000 Hz Transfer function of user external ear
Binaural rendering Real-time processing
High-frequency rendering
Internal ear Acceleration sensor in space (rotation and translation) Dynamic platform
Perception threshold in translation: 5cm/s²
Perception threshold in rotation: 2°/s²
High sensitivity
Tactile Skin sensors Tactile feedback (vibration, electricity,
1000 Hz frequency etc.)
Kinesthesis Muscle sensors Force feedback system
1,000 Hz frequency Treadmill
Walking system
capabilities are not equivalent depending on the Stereoscopic vision given by the two eyes is
luminance. Different physiological research and activated by several coordinated muscles. This

Page 5 of 17
 coordination makes the eyes converge on the ocular distance between each other for a scale 1
object of interest. Because accommodation is rendering.
performed on the object as well, a correlation All the excited senses collaborate during the
between convergence of the two eyes and virtual immersion experience. This collaboration is
accommodation of each eye on object occurs. reflected by a sensory fusion made by the brain to
Thus, foreground and background of the produce information. Depending on the task, the
observed object are blurred on image, which sensory fusion can lead to an average signal
facilitate the fusion between the two left and resulting on several senses or a preference for one
right images and the analysis of the content by sense. A conflict between several modalities can
the brain. The object is then perceived as cause headache or cyber sickness in virtual
unique in the brain. Fusion of the two images of environment. The human is part of the whole
a same object in the brain is more difficult for a system and may adapt his movement depending on
close object. A retinal disparity exists between the system (important latency, for example).
the two images which is accepted by the brain
for small differences. The fusion is still Virtual Reality Issues
possible for images of objects situated in the
area of the main object of attention (called The user is engaged in the immersion process so
Panum area). Accommodation and convergence that his perception results of an alchemy between
contribute to the perception of depth in a scene. his engagement, the technologies, and the task.
Accommodation acts on close objects (less than Thus, the whole system composed of the human
1 m from the point of observation) and and the technology has to be taken into account to
convergence acts on further objects (1–10 m design a virtual reality application. Consequently,
under life conditions). At a distance beyond 10 key factors to consider in the virtual immersion are
m approximately, images from right and left human factors, technologies, and the application as
eyes are quite the same so stereoscopy is not described in Fig. 3. These key factors involve
useful. Activations of each of these two scientific and technological issues jointly
phenomena are correlated. Accommodation addressed by a scientific community focused on
induces a convergence mechanism for a close technology (computer graphics and mechatronics)
object, for example. as well
The visualization of a virtual environment in
stereoscopy requires several steps. A first step is to
create two synthetic images from a point of view
on the virtual environment. Two virtual cameras Human Application
are used from a right and a left virtual eye factors
positions. These two images are rendered and
visualized on a stereoscopic screen with a Virtual
dedicated system to split images between right and immersion
left eyes. Many different systems exist and most of
them use flat screen without any eye convergence
tracking system. So, they do not guarantee the Virtual
correlation between accommodation and reality
technologies
convergence. That is the most important difference
between stereoscopy in real world and stereoscopy
in virtual world. Then, an important disparity as a scientific community on human factors
between images may occur which can cause eye (ergonomics, neuroscience, cognitive science).
discomfort. Previous studies[1] showed that the Sense of presence may be influenced by these
visual system can fusion object images for following main parameters: (a) interactivity of
horizontal parallax less than 1.5° and for vertical the system in the function of the user’s motion,
parallax less than 0.4°. Some rules have to be (b) multisensory feedbacks and quality of the
followed to avoid these side effects. For example, sensory coupling, (c) engagement of the subject
the principal object for the application has to be in the virtual environment, and (d) ecological
situated at the level of the screen. The two virtual interfaces.
cameras used to render the stereoscopic image for
the user have to be situated in parallel axis at inter-

Page 6 of 17
VIRTUAL REALITY TECHNOLOGIES reference of the virtual world with one of the
physical world. Several types of technologies
Interaction between a user and a virtual exist based on mechanical system,
environment requires several different electromagnetism, ultrasound, or image. The
technologies both in the virtual world and in the part of the body which is needed to be tracked
real world. The majority of these technologies in real time depends on the application. The
belong to computer science, human–computer most important part of the body to be tracked is
interfaces, as well as mechatronics. eye position in the physical environment
because vision is often the first sense used by
Sensory Interfaces virtual reality. Eye-tracking technologies can be
used to guarantee the best possible adjustment
Motion Tracking of virtual cameras on each eye (for a good
convergent rendering). But, because of their
Motion tracking technologies are used to complexity and the high speed of eye motion
animate the user’s avatar at the same time and (more than 800°/s), eye-tracking systems are
in the coordination of the user’s motion. These not highly utilized. Most of the time, virtual
technologies aim at informing the computer on reality systems use head tracking instead of eye
the motion of the user. The goal is to adjust the tracking. Right and left eye positions are
estimated
Fig. 3 Key factors for virtual immersion.
from head position and enable to place one virtual camera for each eye to render the virtual image according
to the user’s position in the virtual environment. Another important part of the body to be tracked is the hand
in order to interact with the virtual environment. The choice of the technology depends on the type of
interaction technique (if the interaction technique is more abstract or more natural). For more natural
interaction technique, hand or fingers tracking systems or data glove can be used. For more abstract
interaction technique, several technologies from CAD industry or video games can be used (e.g., joystick,
gyropoint, wand, etc.). If the full body needs to be tracked, dedicated technologies such as real-time full
body–tracking suit, real-time motion-tracking components placed on body joints, and image-based motion
tracking systems exist. These technologies come from the video game industry and entertainment or film
industry.
Visualization Devices sensitive to a very high range of luminance
(from 0.0001 cd/m² to 10,000,000 cd/m²).
Visualization devices use different technologies Unfortunately, actual screens cannot render
to produce image. They can use image image with this requirement (best screens can
projection on screen (based on LCD and lamp render luminance only from 0.001 cd/m² to 500
projector or laser projection) or electronic cd/m²). Dedicated techniques can be used to
screen (based on LCD or OLED technologies). give the user the illusion of a high range of
The characteristics of screen technologies have luminance on image (with the use of high
to be adjusted according to the human eye dynamic range image management system to
capabilities. Because eye resolution is close to generate image and tone-mapping techniques to
1 arcmin, it requires to display image with a render image).
corresponding resolution. For example, a 4 K The first family of visualization system
screen (UHD format) displayed on a 3 m large consists of static screen in front of which the
screen observed at a distance of 2 m is user stands. Screens can be of different shapes
equivalent to a resolution of 1 arcmin from the and sizes. Large screens are useful to render
eye point of view. The human visual system has virtual environment at scale one for user so
a remanence of approximately 30 ms which they can visualize, navigate, and interact in the
requires to produce image at a frame rate of 25 virtual environment as they would do in the real
Hz for monoscopic view (50 Hz for one. They can have the shape of a large wall or
stereoscopic view). But, if the user is in motion a big cube (as the CAVE system) or rounded
in front of the screen (to move around a virtual wall (as the reality center) or a sphere (as
object, for example), fast head motion has to be screen dome). Virtual desk consists of a
considered. Then, video frame rate has to be up horizontal screen at a desk configuration
to 60 Hz for monoscopic view (120 Hz for enabling the user to stand in front of middle-
stereoscopic view). Visual human system is size virtual objects (for design purpose, for
Page 7 of 17
 example). Virtual desk can have an added A third family of visualization system
vertical screen behind the desk to give a better corresponds to volumetric image. One technology
immersion to the user. Small-size screens can consists of volumetric image created by a series of
be used as well for specific purposes. Thus, the images projected through an opto-mechanical
use of PC tablet or smartphone can propose a rotated system. The range of projected images is
virtual window view through the world to give distributed in the space by a rotating screen and
added information to the user in a real or virtual display a volumetric image. Another technology is
world. In using the camera of the device, an constituted by holography. The process was
augmented reality view can be proposed to invented by Denis Gabor in 1947 and consists in
combine the view of real world with elements capturing the phase as well as the amplitude of the
of virtual environment. To render stereoscopic luminous field and restituting it to the user by a
vision using static screen, a dedicated system to laser light. This technique is well known for static
attribute left image to left eye and same for image and several technologies are in progress for
right side is needed. Some of these technologies dynamic images.
filter the light by spectral band or light
polarization. The process invented by Haptic Feedback
d’Almeida in 1858 separates left and right
image flows by the use of a mechanical When interacting with a virtual environment, the
occluding system. Based on this principle, new sense of contact with a virtual object may be
technologies use electronic shutter placed in interesting to render as function of the application.
front of each eye and functioning in switching For that, force feedback technologies may be used.
with the image display. These technologies Technologies come from the robotics domain and
require high-frequency display. The process can take different shapes and complexity. Robot
invented by Berthier in 1896 is composed of a arm uses several joints enabling the robot to follow
physical grid, making the assignment of each the motion of the user’s hand and control this
channel to each eye. New technologies used motion according to collision between the hand’s
this principle to use electronic mask between avatar and the virtual object. Robot can be external
backlight and LCD screen. Another process to the user (the user takes the robot’s arm to control
invented by Bonnet in 1939 uses a lenticular his hand’s avatar) or can be more integrated to the
network placed in front of the screen enabling user (as exoskeleton). To enable the user to
the assignment of channels by the refraction of perceive texture, tactile feedback can be performed
the light through this little lenses. New in using vibrations, pin matrix or pressure
technologies use this principle in front of LCD rendering systems.[2] Some technologies can render
screen. Berthier and Bonnet processes do not heating as well. 3D Sound
need any glasses to render stereoscopic view
unlike polarization or occlusion principles. Theuseof3Dsoundinvirtualrealityapplicationsimpro
The second family of visualization system is vethe immersion of the user in bringing added
constituted of head-mounted display. Screens value when interacting with virtual objects as well
surround eyes and an optical system is used to as audio ambiance in the scenario of the
produce distinct image for the eye. Such application. Because of the difficulty to compute a
technology does not need the system to split right synthetized 3D sound in real time, 3D audio has
and left channels because each eye can directly see been introduced in virtual reality in using existing
its own dedicated image. Two types of head- real sound and spatializing it in virtual
mounted display exist. The first type is closed and environment. Thereafter, it has been used as a
the user sees only the virtual environment. This feedback in case of interaction with the virtual
virtual reality, head-mounted display fully object. The 3D sound has to be precise in order to
immerses the user in the virtual environment as he allow the user to identify the localization of the
cannot see the physical environment. So, they tend sound event in virtual space. Human hearing
to isolate the user from the real world and can depends on several factors as the slight difference
disturb his perception if the other senses are not on perceived sound for both ears in terms of
excited in the same way. A second type of head- intensity (inter-aural level difference) and time
mounted display allows the user to see through the delay (inter-aural
display and combine a virtual image to the view of timedifference),theeffectofhumanbodyandhumanlis
the real scene. This type of device is also called tening system (which can be experimentally
augmented reality device for that reason. measured and modeled by the head-related transfer
Page 8 of 17
function), and the environmental acoustics linear transform, defined by control points, etc.).
(reverberation and reflections of the sound on Nonuniform rational basis splines (NURBS
different parts of the environment). Technologies curves) are equivalent to generalized B-Splines and
to render 3D sound in virtual environment are enable easy manipulation for free-form generation
constituted by real-time head-tracking system, a as well as easy computing of intersections between
headphone system or multichannel objects. This description, commonly used in
systems,andreal- computer-aided design (CAD) domain, is often
timecomputing3Dsoundsystem(software converted in polynomial description and
andhardware).Softwaresdedicatedtoreal- specifically in triangles to facilitate the real-time
timecomputingof 3D sound are based on image- management of the data for graphics rendering. In
rendering principles. To industry, digital mock-up is the central element of
improvetheprecisionofsoundlocalization,soundmet the design process. It is comprehensive and can
aphors can be designed as an added value for the describe geometry (macro and micro), material, as
sound perception. well as assembly operation. Digital mock-up is
involved in the whole life cycle of the product and
Motion used by different actors along the design process
(architecture, thermal analysis, mechanics,
Motion is perceived through several senses such as manufacturing, etc.).
vision, kinesthesis, and vestibular system. Virtual mock-up is dedicated to real-time
Kinesthesis feedbacks consist of on sensors interaction in virtual reality. It is adapted to the
situated on the basis of muscles which feed sensory feedback and the task. For example,
information to the brain on the articulation states. from a unique digital mock-up, two virtual
To excite kinesthesis, virtual reality technologies mock-up might be generated—one virtual
provoke the physical motion of the user. Many mockup dedicated to visualization and another
different systems exist in that goal to make the user one dedicated to haptic feedback. Virtual mock-
walk (treadmill in front of a screen, pressure up is managed by computer. Considering the
sensors on shoes to physically walk when using a actual computer architectures, geometrical data
head-mounted display, virtusphere device, are usually realized on the basis of triangle
circularfloor device) or bicycle (bicycle in place). representation. Computations are simplified but
Vestibular system gives information on numerous. The generation of the virtual mock-
acceleration (in rotation and translation) and can be up from its native digital mock-up is guided by
stimulated with the use of motion platform system such criteria as the application task and user
(commonly used for driving simulation, for interaction. Several methods and tools can be
example). used in that goal as simplification of geometry,
adapted simplification dedicated to sensory
3D Representation feedback, adaptation of the virtual mock-up to
the interaction process, or interaction with the
Data Management lighting. Thus, the transition from digital mock-
up to virtual mock-up consists generally of
Digital mock-up is a set of data representing the transforming the mathematical surface in a set
object to visualize. The virtual object is described of triangles to represent the virtual object.
by its shape, material, and several physical Several existing methods are based on the
properties (e.g., mass, articulations, etc.). Shape Delaunay triangulation or the Voronoi diagram.
description uses techniques of elementary and This transformation induces loss of information
differential geometry. Common techniques on the topology. Depending on the triangulation
represent a virtual object by boundary method, holes may appear in the surface which
representation or constructive solid geometry. The necessitate a specific operation to keep the
use of polygonal plane facets is both simple and coherence of the surface. In function of the
costly because it induces a detailed sampling chosen threshold, the triangulation operation
surface for round objects. Cubic patches enable a can create a representation with too many
fine representation with a relatively low number of triangles to be managed in real time. The virtual
patches to represent an object. B-Spline mock-up has to be simplified then. Several
representation (and particularly Beziers surfaces) methods exist in the literature. A first family
are commonly used because of its interesting consists of suppressing some entities of
features (as continuity capability, invariance to triangles in an iterative way (delete vertices,
Page 9 of 17
fusion of edges or facets). A second family of the virtual object. Texture can restitute details,
consists of modifying the triangles guided by a lighting, as well as roughness of a surface and so
model (surface subdivision, curvature criteria). the details on the image are on the pattern instead
of the 3D model.
Computer Graphics
User Interaction Techniques
Visualization of virtual mock-up is performed
through graphics rendering consisting of creating a Navigation
synthetized image from a user’s point of view. The
created image is the result of a simulation of the Navigation in virtual environment requires
interaction between virtual lighting and virtual devices both for controlling the motion of the
mock-up (characterized by its material and shape). user’s avatar and for perceiving the motion.
Light–material interactions can be very complex to Regarding the application requirement,
represent depending on its features (reflection, different types of technologies can be used. As
refraction, caustic effect, multiple reflections, etc.). mentioned previously, walking or biking
Different models have been developed to express systems can be used. Devices coming from the
these effects.[3] The surface can be modeled by a video game domain (as gamepad) or
succession of micro facets with orientations and motiontracking technologies can also be
sizes function of its features (as roughness). utilized. These navigation devices are used with
Material can also be modeled by the way its dedicated navigation technique to control the
surface modifies the trajectory of the light with the navigation parameters (e.g., speed,
use of a bidirectional reflectance distribution acceleration, translation, rotation). Thus,
function. This function can be expressed by different techniques (more or less intuitive) can
simplified mathematical model or can be be chosen for the same device. Walkingin-place
considered as a scalar for matte surface. To technique proposes the user to mimic the
produce an image, the camera is usually a pin-hole motion of walking when standing in place on a
model. This model enables the simulation of force measurement platform. Grab-the-air is a
trajectory of the light rays from the camera instead technique where the user has to pull the virtual
of lighting to optimize the rendering time. Depth of environment to him by gestures in the air. Gogo
field effect cannot be rendered with the use of the navigation technique is controlling a virtual
pin-hole model and so added signal processing hand in the virtual environment to guide the
may be computed on the image in that goal. Ray direction of navigation. Route-planning is a
tracing technique is used to render the path of the technique where the user points his path in the
light through different materials of the virtual virtual environment and his motion is
scene and can then express reflection and computed according to the path. Target
refraction effects. The simulation principle is quite navigation proposes the user to point the
simple but can take time because of the numerous destination and teleporting him to the
computations to be performed. Depending on the designated location. Virtual companion
complexity of the scene, the materials, and the technique gives the user the control of a virtual
computer’s power, the real-time rendering can be companion to move around in a virtual
challenging with the use of ray-tracing technique. environment. The user’s avatar is attached to
Another complementary method which can render the virtual companion during navigation (by
matte surface lighting effect is the radiosity virtual rope).
technique. This technique is a global illumination Several classifications can be found in the
technique inspired by the domain of thermal literature.[4] Table 2 proposes a classification of
exchange. The scene is decomposed into small different navigation techniques according to two
matte surface elements which exchange lighting different considerations. The first consideration
energy with each other. Because the scene is belongs to the way the action is realized. If the
represented by matte surface elements, the action is realized directly from the avatar, the
luminance at each point of the scene is independent interaction process is made in an egocentric way
from the point of view, making this technique very (coming from the user’s reference). If the action is
useful for real-time visualization after an off-line realized from the virtual environment, the
rendering step. A complementary technique to interaction process is made in an exocentric way
improve the realism of the image is to use texture (external from the user). The second consideration
which consists of applying a pattern on the surface belongs to the nature of the interaction process. In
Page 10 of 17
the case of a natural interaction process, the on object manipulation but necessitates to take care
interaction is considered as concrete. In the case of of compatibility between the techniques used
the interaction process requiring a specific learning during the same task as well as learning
process, this interaction is considered as abstract. requirement.

Manipulation Tactile Interfaces

Manipulating the virtual object is another The large diffusion of tactile interfaces through
important task the user needs to perform in the smartphone and tablet promoted the practice of
virtual environment. The user manipulates the finger interaction language. In touching a sensible
virtual environment to select and change the interface with fingers, a user can interact with a
position of the virtual object or to act on different virtual environment for navigating or
features of the virtual object. As for navigation [Link]
techniques, several manipulation techniques can be mmon for zooming, rotating, moving forward or
used depending on the task and the device. Virtual backward, catching,andsoon. Theseinteraction
hand is a virtual extension of the user’s arm to techniquescanalsobetransposed in a fully
interact (beyond the peripersonal space) in the immersive virtual reality technology.
virtual environment. Virtual pointer enables the
user to reach the virtual object in the virtual space PERCEPTION OF VIRTUAL IMMERSION
as he would do in 2D using mouse control. Ray
casting consists of a virtual ray controlled by the Model of Perception
user
Concept of presence has been proposed as an
Table 2 Navigation techniques. indicator of the virtual immersion since the
Concrete Abstract beginning of virtual reality technology. The
Navigation navigation navigation “Sensorama” virtual theater developed by Morton
characteristics technique technique Heilig in the 1950s introduced the notion of “being
there” in a mediated environment. Several models
Egocentric technique Walking in place Gamepad
of perception have been proposed in the literature
Treadmill Virtual car
in order to define and measure the perception of
Stationary bicycle Grab the air virtual immersion.
Exocentric technique Go-go navigation Virtual The important of these models come from the
companion psychology domain. Bruno Herbelin[7] used the
Target navigation neo-behaviorism psychological theory (where
Route planning the behavior depends on the chain stimulus–
organism–response) to consider the presence as
and interacting with the virtual objects over the
the result of a mediation chain as well as a
distance. Aperture technique enables the user to
psychological mechanism. Franck Biocca[8]
adjust the precision of interaction with the virtual
proposed to consider the body as a
object. Image plane technique is a representation of
communication device included in a
the virtual environment using a 2D projection.
technological chain, mediated by computer.
Voodoo doll technique uses a miniature copy of the
Depending on the extent to which the body is
object to facilitate the interaction process. World-
linked with technology, the human may accept
inminiature replicates the virtual environment in
these technologies as a part of himself (as a
miniature to interact with it. Virtual menu as virtual
cyborg accepting prosthesis). The acceptance
tablet is 3D adaptation of the 2D menu to interact
process depends on the naturalness of
with the object.[5,6] Table 3 proposes a classification
technologies and mind work during the learning
of different manipulation techniques with the same
process. Matthew Lombard[9] defined presence
typology as the one used for navigation techniques.
as what happens when the participant forgets
Of course, different techniques can be combined to
that his perceptions are mediated by
accomplish a task. For example, the user can select
technologies.
the virtual object with one technique and
manipulate it with another one. Using such hybrid
method enables the user a larger degree of freedom

Page 11 of 17
EVALUATION OF VIRTUAL IMMERSION experience can be subjectively evaluated
through questionnaires as the one proposed by
Virtual immersion can be evaluated according Maria Sanchez-Vives[10] on the sense of
to the chosen model of perception. Reactions of presence. In some cases, cyber sickness (caused
the body during or after the virtual immersion by sensory conflict) may occur. Robert
exposure give a first indication on the sense of Kennedy[11] developed a simulator sickness
presence. Physiological feedback or task questionnaire in that context. Determinants of
performance can be objectively measured in presence are numerous. Jonathan Steuer [12]
that objective. Various measurements can be proposed to consider the extent and fidelity of
performed–heart rate, myography activity, sensory information,the real-time synchronism,
encephalography activity, breathing, sweating, content factors (as event, virtual representation
eye motion, speaking, or postural sway. Cyber of the body) and user characteristics
sickness can be evaluated through a (perceptual, cognitive, and motor’s abilities).
combination of objective measurements such as
heart rate, eye movement, and postural sway,
INTERACTION IN VIRTUAL ENVIRONMENT
for example. As well, consequences of the user

Table 3 Manipulation techniques.

Concrete Abstract
Navigation manipulation manipulation
characteristics technique technique
Egocentric Virtual hand Virtual pointer
technique Ray casting
Aperture
technique
Exocentric Image plane technique World in miniature
technique Voodoo doll Virtual menu
Virtual tablet
Naming
Multisensory Interaction

In combining several sensory modalities, the virtual immersion can be improved. The design of a virtual
reality system requires a sufficient knowledge on human senses characteristics for adapting technologies to
application. Depending on the task, a sense can be preferred from another. [13] For example, vision is the main
sense used to join two objects in space for ballistic task and haptic feedback is used for an assembly task
between two parts of an object. Thus, the interaction design process has to take into account the different
tasks of the virtual experience. Multisensory interaction is widely used in the industry domain (e.g., for
design
process, maintenance, training, assembly gestures required by the user to accomplish a
assessment), health domain (e.g., surgery task would be the same in the virtual
training, therapy, gesture training), military environment as in the real world. Interaction
domain (e.g., training), or scientific domain techniques and technologies have to be adapted
(e.g., big data interaction). to these requirements for this goal. A second
type of natural user interaction considers user
experience and common practice with human–
NATURAL USER INTERACTION
computer interaction, as mouse device, joystick
Several interaction technologies and techniques interface, tactile interaction process in using
tend to proposeintuitiveapproachfor theuserand smartphones (with specific interaction language
arecommonly named as natural user based on finger gestures). Natural user
interaction. The first type of natural user interaction is used in the video game domain,
interactionproposes the userto interact industry, entertainment, and health domains
inthesame wayhedoesin the real world. The where users are non-specialists.

Page 12 of 17
NAVIGATION IN VIRTUAL In using head-mounted display, the user can
ENVIRONMENT freely move in the physical space. The user wears a
head-mounted display and a motion-tracking
Exploring the Virtual World system can capture the position of the user in the
space. But, in the real environment where the user
Exploration of the virtual environment is an moves are huge, it is finite because of limitations
important task for a large number of domains of the motion-tracking capabilities. To give the
using virtual reality. Some scientific and illusion of moving in an infinite space, technique
technical challenges remain to restitute motion of redirected walk has been proposed. [16] In this
perception to a user in virtual reality. In the technique, small visual motions in translation and
case of using a static display (the case of large rotation are done continuously in order to replace
screen’ for example), the user cannot make the user in a limited physical area. These small
huge displacements. While the user can move motions are done with speed and acceleration
in real world, he has to stand in front of the which stay under human perception threshold.[17]
screen in virtual reality. So dedicated systems During exploration task in the virtual
have to be proposed in order to produce a environment and depending on technologies,
motion perception. Usually, a compromise has motion sickness may occur. Exploration task is
to be done in the experimental set up design widely used in several application domains such as
between complexity of the technology and virtual building, cultural heritage, big data
motion perception rendering. Walking is one of exploration, etc.
the most natural human motion. The movement
of each human joint during the motion is very Driving
complex as it is correlated to the other joint
movements for a specific task (learning effect) In driving, the user is in interaction with his
under the constraint of gravity. Many vehicle. His own motion is perceived both through
parameters occur during walking which impact visual information by the vection in the peripheral
balance and postural control. Walking motion is field of vision and through the vehicle’s motion
possible in using treadmill technologies.[14] stimulating internal ear. Thus, the sensory
Simple unidirectional treadmill enables the user information comes from the visual channel
to walk along the direction of the treadmill. In combined with vestibular channel. Previous studies
combining several unidirectional treadmills, a showed that a stimulation of the vestibular channel
2D treadmill can be built (some prototypes improves the perception of simulated driving in
exist in research laboratories). As treadmills, limiting the motion sickness. This vestibular
other systems enable the user to walk while channel stimulation is realized by the movement of
staying at the same position relative to the the user with a dynamic platform. Driving
reference of the screen. In the Virtusphere simulation requires a fully virtual immersion
system, the user is inside a sphere rolling on because the motion of the body has to be rendered
site. The user can see the virtual environment finely. While driving, the driver is in contact with
through a head-mounted display. Another the steering wheel, pedal system, and the seat.
system, proposed by Hiroo Iwata,[15] is Thus, sensory feedback as vision, haptic pedal on
constituted of several robots on which the user the feet and steering wheel as well such as
walk and which make him stay static in the vestibular system (acceleration rendering) have to
room. Some drawbacks are present in the use be performed regarding the virtual car behavior
of these systems. Because ofthe inertia ofthe controlled by the driver. If the cockpit is consisted
system, stoppingsuddenly orturning may of a physical device representing a car, the virtual
[Link] environment can only be the road and its
usedwith care (limited speed, low environment. In this case, the driver has a far-
accelerations, safety harness). Another reaching view (more than 10 m) and a stereoscopic
technique uses the principle of walking in rendering is not required. If the cockpit is partially
place. A force sensor placed under the feet (on physical (consisting a steering wheel, pedals, and
shoes) or using a force platform brings seat) and virtual (other parts of the car), the driver
information on alternative pressure under feet. experiments his driving by a far-reaching view
The user is in the front of the screen and (road) and a close view (virtual cockpit), which
realizes movement of trampling, giving him the requires stereoscopic rendering. As the field of
illusion of walking. view has to be wide in order to restitute the vection
Page 13 of 17
effect to the driver, the display has to surround the driving simulators based on different mechanical
driver. Software architecture in driving simulator is systems such as an arm-robot supporting the
constituted by several modules for virtual car driver’s cockpit or centrifuge system to better
simulation, sound rendering, haptic feedback on render accelerations for long time exist.
steering wheel and pedals, and visualization Driving simulator is used for a wide range of
feedback as motion platform. applications. For example, in the industry, driving
Many driving simulators use dynamic platform simulation is used to assess advanced drive
based on the mechanical architecture using parallel assistance system during the design process of a
arms. Parallel platforms have capabilities to vehicle. It can also be used to study the design of
manage important loads while having good semi-automotive vehicles and the cooperation
precision in position. The parallel platform between the user and the vehicle during critical
proposed by Gough and Stewart in 1970 is a situations.
platform operated by six actuators having six In the video game domain, driving simulation is
degrees of freedom. The control of the platform widely used mostly on the basis of static
motion is realized in modifying the length of the simulator and for some with dynamic platform.
actuators. A model of the platform enables to infer Because the driving task stimulates both
inverse cinematic for the control of the platform. visual and vestibular channels, the sensory
Such an inertial platform cannot provide large fusion can be difficult to operate with the use of
longitudinal acceleration due to its architecture. basic technologies, and motion sickness may
The tilt coordination technique is able to restitute occur.
such longitudinal acceleration when using parallel
platform. This technique consists of tilting the
platform compared to the vertical axis. The user VIRTUAL PROTOTYPING
perceives the vertical axis in his own spatial
reference, which is the inertial platform. Thus, in In the industry domain, especially in large
the spatial reference of the platform, gravity can be companies, virtual reality is a tool that is
becoming commonly used in several stages of
decomposed into two components–a component
the product life. Virtual prototyping consists of
perceived as the vertical by the user and a
designing the product, reviewing the product
component perceived as a longitudinal acceleration
with several people, as well as evaluating the
by the user. To prevent the user to perceive the tilt
product in virtual environment. Digital mock-
coordination motion, the inertial platform has to be
up is the core material of virtual prototyping
moved to its tilt coordination position with a tilting
enabling engineers to design the product before
speed below a perception threshold of the
manufacturing it. In the industry (especially
vestibular system. This perception threshold is
small and medium enterprises), evaluation is
estimated at an approximate value of 3°/s. To
performed with physical prototyping, as
facilitate this sensory illusion, the visual perceived
described in Fig. 4. A major issue in the
information has to confirm to the user his position
industry is to evaluate the product based on its
regarding the spatial reference of inertial platform.
virtual representation. Virtual reality is then
For this, visualization has to be computed in
used to enable engineers to take decision in the
accordance with the movements of the platform.
virtual environment as they would do in real
This correlation can be performed by modifying in
world based on physical prototype. Figure 5
real time the image in function of the motion of the
illustrates the validation process in using virtual
platform, if the screen is static. Another solution is
prototyping with
to place the screen on the inertial platform to avoid
real-time computing of the image according to the
platform movement. The control of the platform Real world Virtual world
has to be done in accordance with the perception
Physical
criteria and the degrees of freedom of the platform. prototype
CAD data
Four major motion platform strategies can be
found in the literature towards this. (classic,
adaptive, optimal, predictive). The Gough–Stewart User
platform can be associated to a rail system in order
to extend the degrees of freedom of the whole Evaluation
system (to reproduce lateral acceleration in
overtaking manoeuvres, for example). Other
Page 14 of 17
Fig. 4 Physical prototyping. CONCLUSION

Virtual reality technologies offer a huge amount

Real world Virtual world of possibilities to the user to interact with the
virtual environment. Several competencies are
CAD data necessary to build virtual reality applications
(computer graphics, human computer
Virtual
interfaces, mechatronics, ergonomics, etc.).
User Virtual reality technology developed for a
prototype
dedicated application is specific because each
Virtual application has its own scenario. Thus, virtual
reality reality application is the result of digital
craftsmanship. A very few guidelines exist, and
Evaluation
work-based experience is important to develop
a virtual reality application as a strong
knowledge is required to choose the best fitting
Fig. 5 Virtual prototyping. between technologies and applications.
Nevertheless, some criteria which can lead to a
Virtual world good interaction technique are linked to the
user performance, user comfort, intuitiveness,
and an easy-to-learn interaction technique as
CAD data
well as the effectiveness for the task. Lots of
challenges remain in the future. Some of them
Virtual are related to multisensory augmented reality,
user holography, interaction process with big data or
collaboration over the distance between
asymmetrical virtual reality technologies.
Simulation REFERENCES

1. Fuchs, P.; Moreau, G.; Guitton, P. Virtual Reality:

Concepts and Technologies; CRC Press: Boca
Evaluation
Raton, 2011, 432 pp.
2. Visell, Y.; Duraikkannan, K.A.; Hayward, V. A
device and method for multimodal haptic
rendering of volumetric stiffness. In Haptics:
Fig. 6 Virtual evaluation.
Neuroscience, Devices, Modeling, and
Applications, Part-I; Auvray, M., Duriez, C., Eds.;
Springer: Berlin Heidelberg, 2014; 478–486.
virtual reality technologies compared to the one
3. Foley, J.; van Dam, A.; Feiner, S.; Hughes, J.
done with physical prototype. This process is
Computer Graphics: Principles and Practice;
actually performed in the industry, and some Addison-Wesley: Reading, 1990; 1174 pp.
virtual reality applications provide a more 4. Bowman, D.A.; Kruijff, E.; LaViola, J.J.;
efficient design in terms of time process and Poupyrev, I. 3D User Interfaces: Theory and
evaluation of a multitude of parameters Practice; Addison-Wesley Professional: Redwood
compared to a costly experience plan by City, 2004.
simulation. The added value of virtual reality 5. Dachselt, R.; Hübner, A. Three-dimensional
for prototyping is due to the fact that virtual the menus: A survey and taxonomy. Comput. Graph.
reality process puts the experts in the loop and 2007, 31 (1), 53–65.
the whole process benefits from the expertise 6. Lemoine, P.; Vexo, F.; Thalmann, D. Interaction
of humans. In the future, with more knowledge techniques: 3D menus-based paradigm. In
on the brain and decision-making process, a Workshop on Augmented Virtual Reality, Sept
virtual expert might be able to propose a 2003; University of Geneva: Geneva,
complete virtual evaluation process as Switzerland.
illustrated in Fig. 6. 7. Herbelin, B.; Vexo, F.; Thalmann, D. Sense of
presence in virtual reality exposures therapy. In
1st International Workshop on Virtual Reality

Page 15 of 17
 Rehabilitation, Lausanne, Switzerland, Nov 7–8;
2002.
8. Biocca, F. The cyborg’s dilemma: Progressive
embodiment in virtual environments. J. Comput-
Mediat. Comm. 1997, 3 (2), 12–26.
9. Lombard, M.; Ditton, T. At the heart of it all: The
concept of presence. J. Comput-Mediat. Comm.
1997, 3 (2) online.
10. Sanchez-Vives, M.; Slater, M. From presence to
consciousness through virtual reality. Nat. Rev.
Neurosci. 2005, 6 (4), 332–339.
11. Kennedy, R.; Lane, N.; Berbaum, K.; Lilienthal,
M. Simulator sickness questionnaire: An
enhanced method for quantifying simulator
sickness. Int. J. Aviat. Psychol. 1993, 3 (3), 203–
220.
12. Steuer, J.S. Defining virtual reality: Dimensions
determining telepresence. J. Commun. 1992, 4,
73–93.
13. Ernst, M.O.; Banks, M.S. Humans integrate visual
and haptic information in a statistically optimal
fashion. Nature 2002, 415 (24), 429–433.
14. Souman, J.L.; Giordano, P.R.; Frissen, I.; De
Luca, A.; Ernst, M.O. Making virtual walking
real: Perceptual evaluation of a new treadmill
control algorithm. ACM Trans. Appl. Percept.
2010, 7 (2), 1–14.
15. Iwata, H. Walking about virtual environments on
an infinite floor. In Proceedings of IEEE Virtual
Reality Conference, Houston, TX, Mar 13–17,
1999; IEEE Computer Society: Washington, DC;
286–293.
16. Razzaque, S.; Kohn, Z.; Whitton, M.C.
Redirected walking, In Proceedings of
Eurographics Conference 2001, Manchester, UK,
Sept 2011; 289–294.
17. Peck, T.C.; Fuchs, H.; Whitton, M.C. An
evaluation of navigational ability comparing
Redirected Free Exploration with Distractors to
Walking-in-Place and joystick locomotion
interfaces. In Proceedings of IEEE Virtual Reality
Conference (VR), Mar 19–23, 2011; 55–62.
No index entries found.

Page 16 of 17