Input Devices:
Trackers, Navigation and Gesture Interfaces
Input Devices
What is Virtual Reality?
“A high-end user interface that involves real-time
simulation and interaction through multiple sensorial
channels.” (vision, sound, touch, smell, taste)
Input Devices
Virtual objects have 6 degrees
of freedom ([Link]):
-three translations;
-three rotations.
3-D System of coordinates of a VR object
Input Devices
Trackers measure the motion of “objects” such as user’s
wrist or his head vs. a fixed system of coordinates.
Technologies to perform this task:
Magnetic trackers (used to be prevalent but less used today);
Ultrasonic trackers (least used);
Mechanical trackers (special cases);
Inertial/ultrasonic trackers (new). (volunteers?)
Vision-based trackers (new and geared to be favorite)
GPS
Input Devices
Input Devices
Tracker characteristics:
Measurement rate – Readings/sec;
Sensing latency;
Sensor noise and drift;
Measurement accuracy (errors);
Measurement repeatability;
Tethered or wireless;
Work envelope;
Sensing degradation
Price.
Input Devices
Tracker characteristics:
Accuracy
Resolution
Real object position
Tracker position
measurements
Input Devices
Tracker characteristics:
Tracker data
Real object fixed
position
Signal noise
Time
Input Devices
Tracker characteristics:
Tracker data
Real object fixed Sensor drift
position
Time
Input Devices
Tracker characteristics:
Tracker data
Real object
position
Sensor latency
Time
Input Devices
Tracker characteristics:
Tracker Update Rate
Input Devices
Mechanical Trackers
Definition: A mechanical tracker consists of a serial or
parallel kinematic structure composed of links interconnected
by sensorized joints.
Input Devices
Mechanical Trackers
Pros
Use sensors imbedded in exoskeletons to measure
position;
Have extremely low latencies;
Are immune to interference from magnetic fields and
large metal objects;
Cons
But limit the user’s freedom of motion;
Can be heavy is worn on the body
Input Devices
Push 1280 (Fakespace Inc)
Input Devices
Exoskeleton
structure
Interface
With
computer
Input Devices
Magnetic Trackers
Definition: A magnetic tracker is a non-contact position
measurement device that uses a magnetic field produced
by a usually stationary TRANSMITTER to determine the real-time
position of a usually moving RECEIVER element
Input Devices
Magnetic Trackers
Use low-frequency magnetic fields to measure position;
Fields are produced by a fixed source;
Size of source grows with the tracker work envelope;
Usually the receiver is attached to the tracked object and
has three perpendicular antennas;
Distance is inferred from the voltages induced in the
antennas – needs calibration…
Input Devices
Magnetic tracker with the old Data Glove
Input Devices
Fastrack magnetic tracker system
Electronic interface
Source
Stylus Receiver
Input Devices
Long Ranger source for the tracker system
Source
Receivers
Source
Input Devices
Fastrack magnetic tracker electronics
Metal in lab double ceiling
Polhemus Long Ranger
tracking errors (Rutgers)
Magnitude of Error Vector / Moving Tripod
35.0
Magnitude of Error Vector (inches)
30.0
25.0
Err (54)
20.0 Err (60)
Err (66)
15.0 Err (72)
10.0 Err (80)
5.0
0.0
0 10 20 30 40 50 60 70 80
Transmitter-Receiver Distance (inches)
Tracking error as a function of tripod height
Liberty LATUS – A Polhemus Wireless
Tracker
Uses wireless sources, now called “markers”, each with
a different frequency;
Each source position is measured by a “receptor” within
8 ft (50 sq. ft). 1 receiver can track 4 markers.
The system can have up to 12 markers and up to 16
receptors
Sampling rate is 188 Hz up to 8 markers and
drops to 94 Hz from 9 to 12 markers
Markers are battery powered up to 2.5 hours and weigh
2 ounces each.
For one marker and one receptor accuracy is 0.04 mm
and 0.0012 degree at 1 ft range and drops afterwards.
Communication from base unit is RS232 or USB
Cost – UP TO $64,000
source (marker)
sensor
controller
Input Devices
Motion Star wireless tracker (courtesy of Ascension Technology)
Input Devices
DC Magnetic Tracker Block Diagram
Input Devices Wireless suit (Ascension Technology)
Sensors: 20/suit
100 updates/sec
3 meters range
from base unit
Resolution<2 mm Electronic unit
and <.2 degrees (2 hours battery life)
1.7 kg (3.8 lb)
Input Devices
• Two opposite ERTs 3.63.6 m
• Two side-by-side ERTs 1.8 4.2 m
• Readings weighted to the closer ERTs
• Several Base Stations for several users
Motion Star block diagram
Input Devices
Magnetic Tracker Calibration
Use mechanical measurements to reduce errors;
Sensor noise – variation in measurement with no
real object motion – solved by over-sampling;
Size of errors grow from source outwards;
Errors both in position and orientation.
Input Devices
Magnetic Tracker Errors
due to ambient noise:
e ambient = Kn (d transmitter-receiver)4
due to metal:
Kr (d transmitter-receiver)4
e metal = ---------------------------------
(d transmitter-metal)3 (d metal-receiver)3
Input Devices
Magnetic tracker accuracy degradation
Input Devices
Comparison of AC and DC magnetic trackers
DC trackers are immune to non-ferromagnetic metals
(brass, aluminum and stainless steel)
Both DC and AC trackers are affected by the presence of
Ferromagnetic metals (mild steel and ferrite).
Both are affected by copper;
AC trackers have better resolution and accuracy.
AC trackers have slightly shorter range
Input Devices
Input Devices
Ultrasonic Trackers
Definition: A non-contact position measurement
device that uses an ultrasonic signal produced by a
stationary transmitter to determine the real-time
position/orientation of a moving receiver.
Input Devices
Ultrasonic Trackers
Use low-frequency ultrasound to measure position;
Sound produced by a fixed triangular source (speakers);
Number of sources grows with the tracker work envelope;
The receiver is triangular and attached to the tracked
object and has three microphones;
Distance is inferred from the sound time of flight;
Sensitive to air temperature and other noise sources;
Requires “direct line of sight”;
Slower than magnetic trackers (max 50 updates/sec).
Input Devices
Ultrasonic tracker (Logitech)
Input Devices
Large-volume ultrasonic tracker (Logitech)
Input Devices
Optical Trackers
Definition: A non-contact position measurement
device that uses optical sensing to determine the
real-time position/orientation of an object
Input Devices
Optical trackers: a) outside-looking-in; b) inside-looking-out
Input Devices
Outside-looking-in
LaserBIRD optical tracker
Input Devices
Inside-looking-out
LaserBIRD optical tracker
The two beams are offset
The three vertices are
computed based on known
offset, geometry and angular
velocity of the beams
The average of the three
vertices is computed and sent
to the host at 240 sets/sec
2 ms latency
Range is smaller than that
of magnetic trackers
Outside-looking in: Vicon MX
Uses 4 Mpixel cameras with
own 120 LED array (infrared, or
visible red). Accuracy 0.02 of a
pixel,
Camera has real-time onboard
image processing (masking and
thresholding)
Resolution 2352x1728 @ 160
fps
8 cameras are connected to a
MX net unit which then
communicates with the PC
Input Devices
Outside-looking in: Vicon MX
MX Link connects several MX
Net units for more cameras
To interface with other devices
like a force plate, sensing glove or
eye tracker – use a MX control
unit.
User wears reflective markers
(small spheres).
[Link]
v=xq_nRfoBSm0&feature=related
Input Devices
Outside-looking in: The
Rutgers Arm
Used in rehabilitation
Arm and shoulder
movements are tracked with
active markers and an
infrared camera
Low cost
Input Devices
Inside-out optical tracker advantages
The best accuracy is close to the
work envelope.
Very large tracking surface and
resistance to visual occlusions
(line of sight).
HiBall 3100 wide area tracker
The sensor advantages are:
High sampling rate (2000 Hz, 1000Hz for 2 sensors, 500Hz for 4);
High accuracy (0.4 mm, 0.02°) and high resolution (0.2 mm, 0.03°)
Impervious to metallic or ultrasonic interference;
Very large tracking area (up to 40 ft x 40 ft), small weight (6 oz).
HiBall Optical Sensor HiBall Optical Sensor interior
6 photodiodes
6 optical lenses
Signal conditioning
electronics
(courtesy of 3rdTech Inc.)
HiBall 3000 tracker
on an HMD
Lateral effect
photo diodes
Types of VR Applications
Beacon array modules
(6 strips with 8 LED/strip)
Vision-based tracking for the xBox
The xBox “Project Natal” eliminates the remote.
Natural interaction, and nothing to wear or hold
Uses vision tracking and real-time scene recognition, based in part
software from 3DV Systems, an Israeli company.
Is based on the Time-Of-Flight (TOF) principle, which generates
distance (depth) information for each pixel or object captured by a
camera. Depth data is determined by emitting pulses of infra-red
light to all objects in the scene and sensing the reflected light from
the surface of each object.
xboxprojectnatal's [Link]
Inertial Trackers
Christos Giannopoulos
Introduction
Tracking technologies may be grouped into three
categories: active, passive, and inertial.
While active and passive systems rely on the
occurrence of either produced or naturally occurring
signals, inertial systems are completely self contained,
sensing physical phenomena created by linear
acceleration and angular motion.
How it Works
Inertial sensors determine orientation and position through
Newton's laws of motion. The two devices that are key to
perform inertial tracking are gyroscopes and
accelerometers.
Accelerometers measure linear acceleration vectors with
respect to the inertial reference frame
Gyroscopes measure the rotation rate relative to changes in
orientation.
Properties
Advantages:
• Very low latency
• Prediction based on directly sensed motion derivatives
• High resolution
• Negligible jitter over entire range
• Unrestricted range
• No line-of-sight problems
• Portability: no source to set up
Disadvantages:
• Susceptible to noise & calibration
errors due to gravity field
• Position & orientation drift can
occur
Applications
Due to the versatility of inertial tracking systems, a
multitude of applications for its technology have been
developed including:
• Subterranean Navigation
Inertial navigation technology provides a passive self-contained tracking and
location solution for GPS-denied environments.
• Military Simulation & Training
• Entertainment Applications (“Mo-Cap”)
Utilizes inertial tracking for “motion capturing,” which many 3D animation and
videogame studios use today ([Link]
• Medical Settings
• Oil & Gas Discovery
Sun SPOT System
SPOT - Small Programmable Object Technology
Small wireless battery powered experimental platform
programmed almost entirely in Java, allowing regular
programmers to create projects that used to require
specialized embedded system development skills.
Future Development
Aside from the constant attempt to make these
devices smaller, faster and more precise, an interest in
improving inertial tracker relies on the concept of
Hybrid Tracking.
References
1) [Link]
2) [Link]
3) [Link]
4) Foxlin, Eric. “Miniature 6-DOF inertial system for tracking HMDs”
5) You, Suya. “Hybrid Inertial and Vision Tracking for Augmented Reality
Registration”
6) Lang, Peter. “Inertial Tracking for Mobile Augmented Reality”
7) Wormell, D. “Advanced Inertial-Optical Tracking System for Wide Area
Mixed and Augmented Reality Systems”
8) Bandala, Manuel. “Wireless inertial sensor for tumour motion tracking”
9) [Link]
Input Devices
Hybrid Ultrasonic/Inertial Trackers
No interference from metallic objects;
No interference from magnetic fields;
Large-volume tracking;
“Source-less” orientation tracking;
Full-room tracking;
A newer technology.
Input Devices
But…
Accelerometer errors a lead to decreased
accuracy since x= a t2
2
Errors grow geometrically in time!
Gyroscope errors compound position errors;
Needs independent position estimation to reduce
“drift”;
Input Devices
IS 900 block diagram
VC 2.1- book CD
IS 900 software block diagram
Base unit
Sonic Strips
I-cube
Tracker components (InterSense Co.)
Degrees of freedom: 6
Resolution: 1.5 mm RMS
Angular: 0.05o RMS
Update rate: 180 sets/s max
– one station
Down to 90 updates/sec
- for four stations.
Latency 4–10 ms
Max tracking area: 900 meters2
(300 strips, 24 hubs)
Tracker components (courtesy of Intersense Co.)
I-Cube
Accel./gyro
Ultrasonic
emitter
InterSense Stereo Glasses tracker (courtesy of Intersense Co.)
GPS Trackers and Virtual Reality
Global Navigation Satellite System (GNSS)
- Developed by the US DoD
- Only fully functional GNSS in the world
- Uses 24-32 medium Earth orbit satellites
- Transmits data via microwave signals
Basic Concept of Tracking
- Four or more satellites are used to determine x, y, z,
and t
- GPS satellites broadcast their navigation signals at 50
bit/s
• ~ 65 – 85 milliseconds for the signal to travel
from the satellite to the receiver
- The signal transmits:
• Pseudo-random code
• Ephemeris
• Almanic
- GPS receiver calculates it position by precisely timing
the signals from satellites
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Basic Concept of Tracking (continued)
Typical GPS Receiver Characteristics:
- Refresh Rate:
• Automotive Needs: < 5Hz
• Military Needs: 5Hz to greater than 100Hz
- Accuracy: 5 meters
- Wireless receivers
- Work Envelope: Worldwide, restricted to view of
satellites
- Price: 50 USD - 5000 USD
GPS Error Diagram
1Hz vs. 5 Hz GPS
Receivers
Sony PSP GPS Receiver
Reception Frequency: 1575.42Mhz
Refresh time: Approx. 1 second
Precision: 5 meters
Uses:
• Navigation System
• Stargazing
• Several GPS enabled games
Tourality
- GPS multiplayer game for your mobile phone
- Object is to race to a certain spot before others in the
game
JOYity
- GPS enabled games for the
Android
- YouCatch – GPS enabled
version of ‘Manhunt’
- Roads of San Francisco –
Scavenger Hunt
- City Race Munich – Tourist
trivia game
The GPS Revolution
- iNap
- Shop Savvy
- Google Earth
- Locale
- Goskywatch
- Safety Net
- Sit or Squat
- Twitter
Military operations in Urban
Therein (MOUT)
- Developed at U. New
Orleans
-Uses a web browser with
CosmoPlayer
-Up to 4 team leaders are
rendered in the scene
-Lander et al 2000
Wireless InertiaCube3
up to 4 sensors per receiver
Orientation measurement
180 Hz update rate;
Receiver interfaces via USB or RS232 with
host computer
Wireless receiver Wireless sensor
Hybrid Vision/Inertial Tracker
6 DOF, accuracy 3 mm, 0.1 degree, IS 1200
Latency 2 ms; InertiaCAM
Fixed-focus lens, on-board DSP
Requires a minimum of 1 fiducial Weight 35 grams
Tracking range 10 m per 10 cm
for every 2 m2 at 2 m from camera;
Fiducials
of fiducial diameter
Update rate 120 Hz;
Accelerometer
Ultrasonic
emitter
InterSense Stereo stylus tracker (courtesy of Intersense Co.)
Input Devices
Input Devices
Navigation and Gesture Input Devices
Navigation interfaces allow relative
position control of virtual objects
(including a virtual camera);
Gesture interfaces allow dextrous
control of virtual objects and interaction
through gesture recognition.
Input Devices
Navigation Input Devices
Are the Cubic Mouse, the trackball and the
3-D probe;
Perform relative position/velocity
control of virtual objects;
Allow “fly-by” application by controlling
a virtual camera.
Input Devices
VC 2.2 book CD
The Cubic Mouse
Input Devices
Trackballs
Input Devices
The MicroScribe (Immersion Co.)
Input Devices
Gesture Input Devices
Are sensing gloves such as:
- Fakespace “Pinch Glove”
- 5DT Data Glove;
- The DidjiGlove
- Immersion “CyberGlove”
Have larger work envelope than trackballs/3-D probes;
Need calibration for user’s hand.
Input Devices
Finger Degrees of Freedom
Input Devices
Hand work envelope vs. interface type
The Pinch Glove (Fakespace Co.)
- no joint measures, but contact detection
The Pinch Glove (Fakespace Co.)
P5 Glove
Resistive sensor one per finger
Wrist Tracking
Finger bending resolution 0.5 degrees
0.1 inch wrist tracking resolution at 3 ft range
60 sets/sec $20 to $89
[Link] Movie Link
One optical fiber/finger 5DT Data Glove
A) 5DT Data Glove Ultra
Roll/pitch sensing
100 datasets/sec, 12 bit A/D flexion resolution,
wireless version transmits data at 30 m, needs calibration
The glove interface: a) five-sensor version; b) newer design
5DT Data Glove - continued
Two sensors/finger plus
Two gloves use one Bluetooth 2.4
abduction sensors
GHz transmitter on the user’s belt
The glove interface: a) 14-sensor version; b) wireless kit
5DT Data Glove
Glove has less sensors 5DT Data Glove
than hand joints …
Needs to infer distal
joint flexion angle
The coupling of intermediate and distal finger joints
Input Devices 5DT Data Glove
Input Devices 5DT Data Glove
Linear calibration method
The DG5
glove
Uses one flex sensor/finger;
Angles are obtained by measuring
voltages
100 gestures/sec
Sensor resolution 10 bit (1024 points),
cannot measure individual joints
Needs calibration
less expensive (about $495)
The new DG5
VHand glove 2.0
Uses one bend flex sensor/finger+ accelerometers for position/orientation;
Angles are obtained by measuring voltages
25 gestures/sec
Sensor resolution 10 bit (1024 points),
USB or wireless connection
Incorporates roll, pitch, tracking, 0.5 degree resolution
Needs calibration
less expensive ($585 wired $785 wireless) [Link]
The CyberGlove
Uses 18-22 linear sensors – electrical strain gauges;
Angles are obtained by measuring voltages on
a Wheatstone bridge;
112 gestures/sec “filtered”.
Sensor resolution 0.5 degrees, but errors accumulate
to the fingertip (open kinematic chain);
Sensor repeatability 1 degree
Needs calibration when put on the hand;
Is expensive (about $10,000)
The CyberGlove
(Vertex Co.)
VC 2.3 on book CD
Specifications Pinch P5 Glove 5DT Data Glove DG5 Glove CyberGlove
Glove DG5 VHand
Number of 7/glove 5 5 or 14/glove 5 /glove 18 or
sensors (2 gloves) sensors/glove (1 glove) (1 gloves) 22/glove
Sensor type Electrical (1 glove) Fiber-optic ?(ink film) (1 glove)
Record/sec NA Ink film 100(5DT 5W), 100 Strain gauge
200(5DT 5) 25 (DG 5 VHand) 150(unfiltere
Sensor 1bit 60 8bit 10 bit d)
resolution (2 Points) records/sec (256 Points) (1024 points) 112(filtered)
0.5°
Communicatio Wired Wireless(9.600k Wired (19.2kb)
n Weird b)
(19.2kb) Wireless (DG5
rate None USB 1.0, 2.0 Wired(19.2kb) VHand) Wired
Wrist sensors LED trackers Pitch Accelerometers (115kb)
(5DT 5 model) (DG5 Vhand) Pitch and
yaw