Behavioral and cognitive methods

Mental workload measuring methods

(Indirect methods)
 • The amount of mental resources required to perform a set of
concurrent tasks (Hoedemaeker, 2002).
• Also known as Cognitive Workload.
Mental workload (MWL)
• Sustained high mental workload will cause mental fatigue,
decreased performance and detrimental health effects in the
long run (Holm et al., 2009).
 • MWL mainly describes the input aspects of the
tasks (the requirements, the demands made by the
tasks on the employees).
• MWL is a multidimensional characteristic of task
requirements.
• MWL determines (a) behavior, (b) perceived,
subjective short-term well-being with
consequences for health in long run, (c) psycho-
physiological process.

Image source: [Link]

52353/item/628-mental-workload
 Indirect methods

Mental workload (MWL)

By Eye tracking
By EEG measuring
measuring
Electroencephalogram
 • A widely used non-invasive method for monitoring the brain.
• Based upon placing mental electrodes on the scalp which
measures the small electrical potentials.
• The electrical potentials arise outside of the head due to
neuronal action within the brain.
Introduction
• It has a high time resolution- able to track events within the
brain with millisecond accuracy.
• It is widely used sensing modality for a range of health and
wellbeing applications ranging from epilepsy diagnosis to
emotional monitoring.
 • First carried out in 1929, by the German psychiatrist Hans
Berger.
• Brain activity is characterized by the passing of electrical
impulses along neurons and postsynaptic responses as neurons
Introduction communicate with other one.
• Electrodes attached to the head detect the cumulative electric
field associated with these impulses.
• Potential differences produced can be amplified and stored
giving characteristics representations of brain activity.
 • Brain has a large number of electrical sources present in it:
• Each neurons has intrinsic electrical properties as action
potential are generated by voltage-gated ion-channels in the
cellular membranes.
• Synapses operates based upon the flow of sodium and
Introduction potassium ions.
• The EEG can be viewed as an emergent property of neuronal
and glial cells population and network.
• A voltage waveform with its own characteristic shape and
properties appear on the scalp due to the neuronal action within
the brain.
 • EEG is classified into three components based on field of
application:

Classification
Free running Evoked Hybrid
 • Free running EEG
– The brain activity that is present due to the normal
operation of the brain.
– This EEG is characterized by dividing it into frequency
bands, each given the name:
Free running EEG • Delta (δ) wave- Activity at less than 4 Hz
• Theta (θ) wave- Activity between 4 to 8 Hz
• Alpha (α) wave- Activity between 8 to 13 Hz
• Beta (β) wave- Activity between 13 to 30 Hz
• Gamma (γ) wave- Activity over 30 Hz
 • When a user is restful and closes their eyes
– A dominant alpha (α) rhythm emerges at the back of the
head over the occipital cortex.
• The process of falling asleep is associated with alpha (α)
Free running EEG activity being replaced by slower theta (θ) activity.
• Within free running EEG, there are then a number of features
that occur due to different brain states which will be interest for
different applications.
 • Evoked EEG arises due to stimuli being presented to the user.
• If a user concentrates on a flashing light at a particular
frequency, that stimulus produces a steady-state visual evoked
potential (SSVEP).
– An oscillation at the same frequency as the light source
arises in the EEG at the back of the head.
• Similar steady-state responses can be found due to audio stimuli
Evoked EEG in which case they are termed auditory steady-state response
(ASSR).
• These evoked responses form the fundamental basis of many
brain-computer interface.
• If a screen has multiple light sources, each at a different
frequency, it is possible to tell which source the user is focusing
on as this will be the frequency of the resulting SSVEP.
 • There are a wide number of evoked responses that are possible
due to different forms of stimuli and experimental set ups.
• Event related potentials (ERPs) arise due to the presentation of
individual stimuli, with a gap present before a subsequent
stimuli presentation.
• Some common ERPs are:
– P100- elicited by using checkerboard stimulation.
– N100- produced by the presence of an unexpected
Evoked EEG stimulus, particularly auditory when no other task is being
performed.
– N170- elicited when a face is present in visual stimuli.
– P300- produced by an oddball stimulation, when looked
for uncommon stimuli are observed in a train of other
stimuli.
– N400- produced in response to the recognition of a face
(P = positive going deflection; N = negative going deflection)
 • Following figure shows an example of evoked responses
produced during a face detection task where the user was shown
a series of picture for 1 sec, followed by 1 sec pause, with the
picture randomly selected to be a recognizable face or static
noise.
• A number of evoked responses can be seen- P100, N170, P300.

Evoked EEG
 • This type of EEG is between the free running and evoked EEG.
• No direct stimuli are presented to the user, but they are asked to
think of something or imagine performing an action.
• This can then result in known signal morphologies arising in the
EEG.
• The best known such signal is associated with motor imagery:
• When a user is asked to imagine performing a hand movement,
EEG activity at 8 – 12 and 18 – 26 Hz decreases over the motor
cortex (around the electrode positions C3 and C4)
Hybrid EEG • There is an event-related desynchronization.
 • Typical signals detected by scalp-mounted electrodes are in the
range 1 – 150 μV over a 0.1 – 60 Hz bandwidth.
• The signals both vary temporally and spatially, and so multiple
electrodes are typically used.
• Electrode positions are determined using the 10 – 20 standard,
so named as distances between electrodes are measured as
being 10% or 20% of the skull dimensions (figure in next slide).
Basic set up • A pair of electrodes is required in order to obtain a voltage
potential difference, and each pair of electrode is connected to
an amplifier.
• After suitable amplification and bandwidth limiting, the signals
are stored in a suitable location.
• The devices digitize the signals allowing them to be stored,
wirelessly transmitted or analyzed in real time
Basic set up

The standard 10 – 20 electrode system for electrode placement and names

 • A user with a head cap on which has holes to hold a number of
electrodes next to the scalp.
• On each electrode a conductive gel is placed in order to ensure
that a good contact is made between the metal of the electrode
and the scalp.
• This gel provides a good electrical contact with the head, and it
can act as a mechanical buffer to ensure that the connection is
maintained even during and after head movement.
Practical set up

A conventional EEG set up with electrode on the scalp

 • A conceptual set up of EEG recording instrumentation.
• Electrodes placed on the scalp detect s small electrical signal
which is amplified and stored.
• A recording the from the electrodes forms one EEG channel,
and different montages are possible depending on which
electrodes are used.
• Modern set ups favor referential recording as this allows any
desired other montage to be derived offline by software.
Practical set up

Storage or wireless
Different amplifier Analogue to digital
communication as
V1 Volt = GD (V1 – V2) conversion
desired

V2
Common mode feedback
Vcm = Gc (V1 + V2)
Head

A simplified overview of an EEG set up

 Summarized standards for clinical-grade digital EEG recording

Interchannel crosstalk less than 1% (40dB

At least 24 channels, preferably 32
down or better)

At least 12 bit analogue-to-digital conversion 70 Hz, 40 dB per decade, anti-aliasing filter

with a minimum resolution of 0.5 μV required

Minimum sampling rate 200 Hz, preferably Recording noise not more than 1.5 μVpp or 0.5
higher μVrms
Practical set up
Common-mode rejection ratio at amplifier
High pass filtering at 0.16 Hz or less
input at least 110 dB

50/ 60 Hz notch filter (not specified Q) Referential montage to allow later re-
available but not routinely used montaging

Electrode impedances below 5 kΩ Amplifier impedances over 100 MΩ

 Data epoch Motion artifact
Input data
selection removal

Signal processing and data

analysis Feature extraction
Output

Classifier Normalization Feature selection

Basic EEG signal processing

 • EEG data is the input to the signal processing chain.
• Motion artifacts- when the user’s own movements affect the
placement of the sensor or its contact pressure to the skin.
• Motion artifacts can be minimized at the data collection stage
by ensuring that the electrodes are correctly and well connected
to the head.
• Most approaches for removing such affect and recovering clean
Signal processing and data EEG data are based upon signal decomposition technique;
analysis independent component analysis (ICA) and principle
component analysis (PCA).
• Independent component analysis (ICA)- allows artifacts
removal to be performed without having to be involved with the
full mathematical details.
• Principle component analysis (PCA)- its mathematical
conditioning is dependent on the number of EEG channels
present.
 • Given the EEG input, are the same as used in data-driven
machine learning approach.
• The end aim is to use a classifier to make an end decision about
the current window of data being analyzed.
• This might be to indicate the presence of an evoked response or
Signal processing and data to classify a section of data as being associated with a high
analysis workload or a low workload.
• To do this a number of features are calculated from the data.
• A list of such 65 features is given and it is also recognized that
non-linear mathematics can provide many methods for
discriminating points of interest from background EEG and
residual artifacts.
 • Finally, attention must be given to the performance assessment
Signal processing and data methodologies for EEG signal processing.
analysis • Applying signal processing to the EEG is straight forwards.
 • Electrodes
• 3 types of electrodes are available- passive wet, active and dry.
– Passive wet EEG electrode
– Disposable and reusable forms
– There is a hole in the centre of electrode to allow a
conductive gel to added which ensures a conductive path
is made between the electrode metal and scalp.
– It is a transducer which converts ionic current coming
Modality of measurements from human body into electron currents that can be
measured by conventional electronics.
 • Active EEG electrode
• A buffer amplifier is placed on top of the electrode itself.
• It reduces main interference and artifacts due to movement of
recording wires.
• If a buffer amplifier is included, the cable is connected to the
low-output impedance buffer and the same amount of induced
current produces a much smaller interfering voltage.
• The high impedance node between the amplifier and the
Modality of measurements electrode is made to be much shorter minimizing the path for
interference to be introduced.
 • Dry electrodes
• No need to have a conductive gel.
• This electrode is based on a 3mm steel disc coated with nitride
on one side and an impedance-converting amplifier on the other.
• It is much more difficult to keep the electrodes in place and next
to the scalp for long-term recordings.
• Most dry electrodes having fingers to better penetrate the hair.
• These are attached to springs to help keep them place.

Modality of measurements
 • Instrumentation
• Following figure shows the evaluation of EEG instrumentation.
• These are basically 4 types-
– Research Lab/ Inpatient
– Ambulatory
– Wearable
– Beyond wearable

Modality of measurements
 • Research Lab/ Inpatient
• Large, non-portable and very high quality, flexible in their use.
• Can be use for anything from a standard short EEG test for
clinical use which lasts between 20 to 30 min, upto a few hours
for experiments in a research laboratory.
• Also used for sleep testing.
• The electrode set up is typically passive wet EEG electrodes,
with long wires connected the electrodes to an amplifier box.

Modality of measurements
 • Ambulatory (AEEG)
• This might be worn on the back of the head or on a belt lower
down the body.
• Only a few EEG channels (3 – 4) would be recorded on to a
cassette tape.
• It can be used as an outpatient monitoring arrangement, which
is cheaper than inpatient monitoring.
• It also allows the patient to be monitored in their natural
environment, of use for research studies where the subject
Modality of measurements might be biased by being in a lab environment that they are not
familiar with.
 • Wearable
• It creates smaller and more discrete EEG units that are quick
and easy to set up by non-trained users.
• It is normally based upon using headsets with arms that wrap
around the head, rather than needing a full head cap.
• Focus on to low channels counts allowing quicker set ups and
the collection of information at key point around the head,
rather than full head montage.
• Example- Emotiv, Muse, Neurosky
Modality of measurements
 • Beyond wearable
• It uses a hybrid approach of tattoos for non-haired regions and
fingered electrodes for hair regions, or only non-haired EEG
with signal processing used to project the ear or forehead based
signals onto classical electrodes positions.
• Different types of approaches of EEG are present- Forehead,
Ear approaches

Modality of measurements
EPOC Emotive Device (EEG)
 • EPOC Emotiv device is created by the Emotiv Inc.
• Extensively used in Brain-Computer Interface (BCI)
applications.
• It provides an excellent balance between price and quality, with
the additional benefit that it can be used wirelessly.
Introduction • It is brief and intended to be read and used by anyone who
wanted to start working with this device.
• Additionally, it details how to use the standard APIs and its
usage from MATLAB and a basic analysis by using EEGLAB.
• It describes how to use Emokit, an open source version of the
USB dongle driver, and how to read the signals from python.
 Neuroheadset

Electrodes

EPOC Emotiv Neuroheadset USB dongle driver

Saline

Emotiv software
 • The device contains 16 electrode positions to provide a good-
enough 14 channel’s EEG signal.
• The device fits like a headset, and each electrode fits into their
aimed position by a plastic and flexible lever.
• Hairless subjects are easier to work with.

EPOC Emotiv Neuroheadset

Neuroheadset
 • Total 16 electrodes are contained in the EPOC Emotiv box.
• A large hydrator pad is present inside back of the cover.
• Each electrode is covered with a soft hydrator pod on the top,
which is needed to be hydrated with Saline.
• Few drops of saline is needed to hydrate the inside back
hydrator pad.
• This hydration maintain the moisture of the pads of electrodes
EPOC Emotiv Neuroheadset when they are not used.

Hydrator pad Hydration of pad

Electrode covered on the top

with Hydrator pad
 • The device comes additionally with a USB dongle which works
with a proprietary UHF communication protocol.
• When this dingle is plugged in and the device is turned on, it
will be ready to transmit all the data packages directly to the
host computer.

EPOC Emotiv Neuroheadset

USB Transceiver dongle

 • Emotiv Control Panel program is installed on a computer.
• It is used to measure the impedance of each electrode.
• This displays indicators that provide real-time information
about EmoEngine status and EPOC Neuroheadset sensor
contact quality.
• It also exposes user profile management controls.

EPOC Emotiv Neuroheadset

Emotiv Control Panel screen

 • The headset contains 16 plastic electrode slots.
• Each slot is a plastic cylinder which contains a golden electrode
plate at the bottom of it.
• Small pads dumped with saline solution can be inserted on
small plastic cases that can be screwed into each slot.
• These plastic cases can be bought separately and the entire set
of electrodes can be replaced.

Setup

Electrode placed on slot

 • Insert the supplied USB Transceiver dongle into one the USB
slot’s.
• Use a USB extension cable and position the Transceiver in
prominent location away from monitor and PC to improve poor
reception.
• Then turn on the headset using the switch at the bottom end of
the headset, holding it close to the Transceiver.
Setup

Insert the USB Transceiver dongle Turn on device and connect to dongle
 Notice 1

• USB dongle works much better when it is attached to a USB extension cable,
located around 50 cm from the computer.

Notice 2

Important notice • The electrode pads should be lightly damped in saline solution
• As more dumped they are, the connection is actually better.

Notice 3

• The box where the electrodes are stored, contains a bigger pad that can be
also dumped in saline solution.
• This will really helps to have the electrode pads wet enough for the next
recording season.
• Otherwise they get dry and turn very rigid and it is much more difficult to
damp them again to work properly.
 • The pads must be first separately from their plastic cases.
• After damped them with enough saline solution they can be
inserted into each electrode slot of headset, verifying they fit
Wearing the device deeply enough to touch the bottom of the electrode slot.
• Each one of the electrode can be screwed, tightly inside each
one of the electrode cases of the device.
 • Using both hands, slide the headset down from the top of the
head.
• Place the arms approximately as depicted, being careful to place
the sensors with the black rubber insert on the bone just behind
each ear lobe.
• Correct placement of the rubber sensor is critical for correct
operation.

Wearing the device

 • The 2 front sensors should be approximately at the hairline or
about the width of 3 fingers above the eyebrows.

Wearing the device

 • After headset is in position, press and hold the 2 reference
sensors (located just above and behind the ears) for about 5 – 10
seconds.
• Once USB dingle is connected, a green light start to blink.
• Good contact of reference sensors is the key for a good signal.
• Check that the light corresponding to these 2 reference sensors
turn from red to green in the EPOC Control Panel Setup screen.

Wearing the device

 • Gently press and hold each remaining sensor against scalp until
all the lights corresponding to those sensors turn to green in the
EPOC Control Panel.
• If the user is unable to get anything expect 2 red sensors, add
saline to the reference and other sensors, or try the alternate
reference locations- swap the reference sensors with the rubber
comfort pads located directly behind the ears, making sure the
reference sensors contact directly onto the bare skin on the bony
bump.

Wearing the device

No connection The color indicates different

All connections are good
state of signal
 • The objective is to achieve as many green as possible using the
EPOC Control Panel and adjusting the position of the various
arms accordingly.
• EPOC will still function with some sensor locations showing
yellow or orange, and will even cope with a few red or black
however the detections will be less reliable in this state.

Signal quality screen

 • The top pane of EPOC Control Panel is known as EmoEngine
Status Panel.
• This pane displays indicators that provide real-time information
about EmoEngine status and EPOC Neuroheadset sensor
contact quality.
EmoEngine Status Panel
• It also exposes user profile management control.

EmoEngine Status Panel

 • System status: A summary of the general EmoEngine status.
• System Up Time: The timestamp (in second) attached to the
most recently received EmoState event. Generally, this
corresponds to the length of time that the EmoEngine has been
running with an EPOC Neuroheadset connected to the USB
receiver.
• Wireless signal: This displays the quality of the connection
between the EPOC Neuroheadset and the Emotiv wireless USB
receiver connected to machine.
EmoEngine Status Panel • Battery power: Display an approximation of the remaining
charge in an EPOC Neuroheadset’s built-in battery.

EmoEngine Status Panel

 • Use the controls to manage user profiles and assign a specific
user to a specific attached EPOC Neuroheadset.
• EPOC Control Panel only displays status information and
detection results for a single EPOC Neuroheadset at a time.

User Status

Add new user profile

 • The head setup panel is displayed by default when starting
EPOC Control Panel.
• The color of sensors circle is a representation of the contact
quality.
– Black color- No signal (not acceptable)
– Red color- very poor signal (not acceptable)
– Orange color- Poor signal
– Yellow color- Fair signal
– Green color- Good signal
Neuroheadset up

Neuroheadset Setup Panel

 • It details the facial expressions and non-verbal communications
capabilities of the EPOC Neuroheadset.
• Now, if you want to show facial expressions on your avatar, you
need only perform them while wearing the EPOC
Neuroheadset.
• You can display these expressions on the face of your avatar.

Expressive™ Suite
 • On the left hand side of the panel, the avatar will mimic your
facial expressions, in camera view.
• On the right hand side of the panel is the Sensitivity Panel.

Expressive™ Suite
 • The Sensitivity Panel offers sensitivity adjustments for the
Expressive Suite detections.
• For each facial expression, check the performance of the
detection.
• Sensitivity can be increased or decreased by moving the
sensitivity slider to the right or left, respectively.

Expressive™ Suite
 • It measures and displays a wide range of subjective emotional
responses.
• Levels of interest, excitement, and engagement that experienced
in a cyber environment are translated into graphical
measurements on the control panel dials, and in some cases,
into dynamically changing environments within games are
played.

Affective™ Suite
 • It reports real time changes in the subjective emotions
experienced by the user.
• It currently displays 3 short-term and 3 long-term Affective
detections: Meditation, Engagement, Excitement.
• Emotions related to Engagement are alertness, vigilance,
concentration, stimulation, interest.

Affective™ Suite
 • It contains two graph panes displaying the 3 Affective
detections instantaneously and long-term average time scales.
• The top chart is configured to plot 30 sec of data for current and
the bottom chart is plotted to display 5 min worth of data for
long-term.

Affective™ Suite
 • Instantaneous Excitement is experienced as an awareness or
feeling of physiological arousal with a positive value.
• Related emotions: titillation, nervousness, agitation.
• Long-term excitement is experienced and defined in the same
way as Instantaneous.

Affective™ Suite
 • It uses a virtual 3D cube to display an animated representation
of the Cognitive detection output.
• It evaluates a user’s real time brainwave activity to discern the
user’s conscious intent to perform distinct physical actions on a
real or virtual object.
• The detection is designed to work with upto 13 different action:
6 directional movements (push, pull, left, right, up, down) and 6
rotations (clockwise, counter clockwise, left, right, forward,
backward) and 1 additional action that exists only in the realm
of the user’s imagination: disappear.

Cognitive™ Suite
 • The 3D cube assists the user in visualizing the intended action
during the training process.
• The information on the right panel displays the current state of
cognitive detection and allows the user to define the current set
of action.
• A green checkmark is used to indicate that the corresponding
action has been trained.
• An orange ‘X’ indicates a lack of training data.
• Pressing the train button, the cognitive training process will
start.
Cognitive™ Suite
 • First, train a neutral state.
• Simply relax, act natural and clear the mind.
• Press TRAIN button to bring up the training screen.
• Press GO when you are ready.
• A progress bar will appear as your brainwave are observed.
• Once training is completed, you will be asked to accept or reject
the training session by pressing YES or NO.
• Once you accept, you will be automatically returned to the
Cognitive Panel Display.
Cognitive™ Suite
 • Non-directly medical applications of EEG technology
• Brain-computer interface
• It allows the operation of a computer to be changed or adopted
Applications without having to use standard interfacing methods.
• Other application for assisting subjects who are paralyzed and
who might not be able to control a computer or a powered
wheelchair using conventional interface.
 • The EEG has its core applications in medical diagnosis.
• But many different brain-computer interface and consumer
Summary neuroscience applications can be investigated.
Eye Tracking
 • Eye tracking- A technique that can measure eye movement
activity.
• It actually measures visual activity on a given stimuli.
• It gives information about
– Where do we look?
Introduction – What is ignored?
– How the pupil reacts to different stimuli?
• It can also analyzes the human behavior by following eye
movements.
• Eye tracking relates to cognitive processes.
Anatomy of Human Eye
 • Pupil- Controls the amount of light that can enter the eye.
• Cornea- Together with a sclera, forms a protective layer of an eye
and two photoreceptors.
• Retina- The light sensitive layer of photoreceptors at the back of the
eye. It receives image and sends them as electrical signals through
the optic nerve to the brain.
• Photoreceptors- A type of cell which receives the light stimuli and
interpret into neural stimuli to the brain
– Rods- Responsible for peripheral vision and sensitive to
objects movement.
– Cones- Provide the central vision. Do well in bright light and
Anatomy of Human Eye discriminate colour.
 Video clips

Type of stimuli Images

Texts
 • Fixation- A semi stationary movement on a specific area.
• Saccade- The rapid movements between fixations.
• Regression- A reverse saccade.
• Interest area- An area on which the eye movement is more
Eye movements than other areas.

Saccade Regression Interest area

2 6 7 9
8
10

The knight attacked the windmill on his donkey 5

1 3 4

Fixation
 • Place the information that interest us on the fovea-
– A fixation
– Saccade
• Fovea- A small depression within the neurosensory retina,
where visual acuity is the highest.
• Keep the image stationary on the retina in spite of movements
of objects or one’s head by a smooth pursuit (used to follow a
Main functions of eye moving target).
movement • Prevent stationary objects from fading perceptually.
– Micro-saccades: A kind of fixation eye movement. They
are small, jerk-like, involuntary eye movement. It occurs
during prolonged visual fixation.
– Tremors: A constant, involuntary eye movement
– Drifts: A slow eye movement that is occurred between
micro saccades during attempted fixation.
 • The general use of real time eye movement data was relatively
less focused in Human Computer Interaction (HCI).
• In 1999, Kyualg Nam Kim and S.R. Ramakrishnan proposed a
vision based eye gaze tracking method used for HCI.
Eye tracking technology • They proposed eye gaze as input mode for efficient computer
interface.
• Eye movements were the focus of research in this eye gaze
tracking method.
 • Gaze tracking is typically employed to determine a person’s
focus of attention.
• Eye movement provides a rich and informative window into a
person’s thought and intentions.
• Gaze- Point of regard.
• Gaze tracking is an analysis of eye tracking data with respect to
the head/ visual scene.
Eye gaze tracking • Gaze tracking mainly can be used in following ways:
– Cognitive and behavioural therapy
– Visual search
– Marketing/ advertising domain
– Neuroscience
– Psychology
– Human Computer Interaction (HCI)
 • The integration of eye and head position is used to compute the
location of gaze in the visual scene.
• Simple eye tracker reports only the direction of the gaze relative
to the head (with head-mounted system, electrodes, sceleral
coils) or for a fixed position of the eyeball (system which
requires a head fixation).
Eye gaze tracking • Such eye tracking systems are referred as intrusive or invasive
systems because some special contacting devices are attached to
the skin or eye to catch the user’s gaze.
• The system which do not have any physical contact with user
and eye tracker apparatus are referred as non-intrusive systems
or remote systems.
 • Eye tracker basically captures eye movement by using light
source that illuminates eye that causes visible reflection.
• It uses a high resolution camera to capture eye image to show
these reflections.
• Then this eye image is used to identify the reflections of the
light source on the cornea and pupil.
• Then vector formed by angle between the cornea and pupil
reflections is calculated.
• Based on this information the gaze direction is calculated.
• The vector calculation methods include:
– Velocity based
Eye gaze tracking
– Dispersion based
– Area of interest
 • Eye tracker type
• The following two types of eye trackers are available in market:

Remote/ screen
based eye tracker

Eye gaze tracking

Head mounted eye

tracker
 • Remote/ screen based eye tracker
• This type of eye tracker requires the respondents/ participants to
sit in front of a screen to interact with the stimuli or screen
based content.
• Remote ET system track the eyes within certain limits called
headbox, but the eye movement freedom is sufficiently large
and the respondents feel unrestricted .
Eye gaze tracking • Following are the characteristics of remote eye tracker:
– Records eye movement at a distance.
– There are no attachments to the respondents.
– Computer or screen mounted.
– Respondents sits in front of ET.
– Observations on any screen-based stimulus materials,
offline stimuli can be recorded.

Remote/ screen based eye tracker

 • Head-mounted or mobile Eye Tracker
• These are fitted near the eyes and allow respondent/ participants
to move freely.
• These are used if your study need to perform tasks in a natural
environment.
• Glasses might shift during the recording.
• Following are the characteristics of mobile Eye tracker:
– Records eye movement from a close range.
– Mounted on a lightweight eyeglasses.
Eye gaze tracking – Respondent can walk around freely.

Head mounted/ mobile eye tracker

 • The eye trackers are available in the market from following
organizations:
– Tobii
– SMIVision
– EyeLink
– Interactive Minds
– Imotions
Tobii
– Mirametrics
Eye gaze tracking
– EyeTech

SMIVision

Mirametrics
EyeLink EyeTech
 • Following metrics from an eye tracker and the information can
be analyzed for various applications:

Gaze points Fixation Smooth pursuit

Saccades Scan paths Heat maps

Eye tracker metrics

Area of interest Fixation sequence Respondent count

Time spent
 • Fixation and gaze points
• The main metrics used in eye tracking are fixations and gaze
points
– Gaze points
– The basic units of measuring the eye movements are gaze
points.
– Each gaze point is an individual record of the participant’s
gaze at a given moment.
– The number of this individual moments per second
Eye tracker metrics depends on the sampling rate of ET device.
– One gaze point is one row captured by the ET device.
 • Fixation and gaze points
• The main metrics used in eye tracking are fixations and gaze
points
– Fixation
– A fixation is a cluster denoted by a series of gaze points
which are to be close in time and range.
– Fixation is a period in which our eyes are fixed at a
particular object in a stimulus.
Eye tracker metrics – Typically fixation duration is 100 – 300 millisecond.
 • Smooth pursuit
• It is an eye movement that allows eyes to closely follow a
moving object, i.e., voluntarily shift gaze.
• Example- imagine watching clouds in the sky.
• Here our eye movements are quite opposite as our eyes follow
moving object steadily (clouds)

Eye tracker metrics

 • Saccades
• A rapid eye movements which redirects a visual axis to a new
location.
• The rapid eye movements between fixations are referred to as a
saccades.
• Example- Reading a book our eye movements are not smooth
across the line but instead our eyes jump and pause generating a
number of saccades.
• Typically saccades are measured in angle velocity.
Eye tracker metrics • On an average, saccades span 7 – 9 characters along a line that
contains text.
• This can be used to study reading behaviour as early or expert
leaders.
Saccade
2

The knight attacked 3

1

Fixation
 • Scanpath
• It was first defined by David Noton and Lawrence Stark in
1971.
• The sequence of fixation-saccade-fixation is referred to as
scanpath.
• It is an ordered set of fixations points (depicted by circle)
connected by saccades (depicted by lines).

Eye tracker metrics

 • Heat maps
• The static or dynamic or static aggregations of gaze points and
fixations generate the distribution of visual attention are
presented through the heat map.
• Heat maps serve as an excellent method for visualization; it
shows maximum attention area of the stimulus.
• Heat maps use easy to read colour coded scheme.
• A high number of gaze points are indicated by Red area and
show an increased level of interest.
Eye tracker metrics • Yellow and green area point toward less visual attention.

High number of gaze points

Low number of gaze points

Medium number of gaze points

 • Area of interest
• Area of interests (AOI) are sub regions of a stimulus object
displayed on screen defined by user.
• AOIs is defined by more fixation and saccades on a particular
area.
• Metrics to separate AOIs are evaluated with the performance of
two or more specific areas in the same picture, website or any
program interface.

Eye tracker metrics

 • Fixation sequence
• It can be generated based on fixation position and time
information.
• It depends on where a respondent looks and for how long.
• We can build an order of attention where the respondent looked
first, second and so on.
• This parameter is used in research as it reflects salient elements
in the display or in an environment that catch much attention.
• AOIs that respondents look at first are visually more salient and
Eye tracker metrics hence are of more interest.

ms = Millisecond
 • Respondent count
• This metric allows to extract more information about the
number of respondents had gaze direction towards a specific
AOIs.
• Higher respondents count indicate that fixations and gaze points
are driven by some external aspects in the stimulus.

Eye tracker metrics

 • Time spent
• Time spent specifies the amount of time that respondents have
Eye tracker metrics spent on a specific AOI.
• It often indicates motivation and conscious attention because
long prevalence at a region to a high level interest.
 • Each eye tracker has at least two components:
– A light source
– A camera
• The light source (usually infrared) is directed toward the eye.
• The camera tracks the reflection of the light source along with
visible ocular features such as pupil.
• This data is used to extrapolate the rotation of the eye and
ultimately the direction of gaze.
Basic procedure • Additional information such as blink frequency and changes in
pupil diameter are also detected by the eye tracker.
 • Eye trackers collect raw eye movement data points every 16.6
or 8.3 ms (depending whether the sampling data rate is 60Hz or
120 Hz).
• Each data point will be identified by a ‘timestamp’ and ‘x,y’
coordinates, and sent to the analysis application database
running on the computer connected to the eye tracker.
• In order to visualize the data these coordinates will then be
processed further into fixations and overlaid on a video
Basic procedure recording of the stimuli used in the test.
 Prepare

• Project ID, subject ID, subject calibration

Record

Eye tracking stages • Instructions, stimulus, preview, closure

Analyze

• AOI- Defined area on a stimulus at which analysis of eye activity

is made
• Gaze plot- Eye movements sequence, order and duration of gaze
fixation of individual subject
• Heat maps- Qualitative analysis of visual activity by colour
 Electro Oculography

Sceleral Search Coil

Different methods
Infrared Oculography

Video Oculography
 • In this method, sensors are attached at the skin around the eyes
to measure an electric field exists when eyes rotate.
• By recording small differences in the skin potential around the
eye, the position of the eye can be estimated.

Electro Oculography
 • When a coil of wire moves in a magnetic field, the field induces
a voltage in the coil.
• If the coil is attached to the eye, then a signal of eye position
will be produced.
• In order to measure human eye movement, small coils of wire
are embedded in a modified contact lens.
• This is inserted into the eye after local anaesthetics has been
introduced.
Sceleral Search Coil • An integrated mirror in the contact lens allows measuring
reflected light.
 • It measures intensity of reflected light.
• In this method, the eye is illuminated by infrared light which is
reflected by the sclera.
• The light source and sensors can be placed on spherical glasses.

Infrared Oculography
 • It is most widely used eye tracking method in commercial eye
trackers.
• It uses single or multiple cameras to determine the movement of
eye using the information obtained form the images captured.

Video Oculography
SMI Eye Tracking & SMI BeGaze
 • The SMI Experiment Centre™, with SMI eye tracking glass
and SMI BeGaze™ software are a versatile suite designed to
address the various requirements in the eye tracking world.
• Designed particularly for researches working in the fields of
Introduction marketing research, usability testing, psychology, neurology
and cognitive neuroscience, this platform supports a broad
range of studies.
• It is ideal for evaluating interactive media, e.g., website,
software, image, print and online advertising.
 SMI Eye Tracking Glass

SMI Experiment Centre™

Tools

SMI BeGaze™ software

 • SMI Eye Tracking Glasses 2 Wireless (SMI ETG 2w) are
designed to capture a person’s natural gaze behaviour in real-
time.
• It provides robustness, mobility and ease of use for a broad
range of real-time tasks.
• The Natural Gaze™ design of the head gear for SMI ETG 2w is
designed for natural recording of gaze behaviour.
• It provides maximal peripheral perception and binocular vision-
important for realistic depth perception and natural visual
SMI Eye Tracking Glass orientation.
 • It is designed to make gaze tracking experiments and visual
stimuli creation a snap.
• Experiment centre features a user friendly intuitive interface.
• All steps for preparing, testing and running an experiment are
executed in the application window.
• It delivers experiment design and experiment control which
allows to prepare and execute gaze tracking experiment with
following features:
– A single user interface for managing various functions
SMI Experiment Centre™ (stimulus preview, live gaze monitoring, precision timing)
– Calibration and Validation on demand.
 • BeGaze provide quick and comprehensive analysis and
visualization on eye tracking data and stimuli.
• BeGaze greatly simplifies monocular and binocular tracking
data analysis by structuring the information on stimuli and
subjects, as well as displaying the result as meaningful graphs
and visuals- all in one advanced applications.

SMI BeGaze™ software

 • The results from BeGaze:
– Visual attention analysis with scan path, heat map diagram
on still and dynamic stimuli.
– Visual search pattern analysis based on user-defined areas
of interest, also dynamically changing or video.
SMI BeGaze™ software – Quantitative analysis of saccades, fixations, blinks and
reaction times to user-defined events.
– Video and bitmap export of scanpath and attentions map.
– Fully configurable statistics export with grouping and
sorting functions.
 • Some definition of icons in BeGaze window:

SMI BeGaze™ software

 • Some definition of icons in BeGaze window:

SMI BeGaze™ software

 Design/ Prepare

Procedures Record

Analysis
 • Design/ Prepare
– When designing an experiment, determine the participant
properties and questionnaires to collect during the study
Procedures and choose calibration and hardware settings.
– Set up the SMI ETG with the participants and perform a
calibration/ validation.
 • Record
– You have three option to record the scene video and eye
tracking data with SMI Eye Tracking Glasses.
– Use the recording unit for fully mobile data collection.
Procedures
– Choose the SMI-ETG Laptop for live observation and
real-time data access via the SMI SDK, or combine the
advantages of the SMI-ETG Laptop and the Recording
Unit for fully flexibility.
 • Analysis
– Watch your scene videos with gaze overlay in SMI
BeGaze analysis software.
– Use SMI Semantic Gaze Mapping to efficiently quantify
Procedures and visualize consolidated eye tracking data taken from
multiple participants on target areas.
– Export statistics and visualizations like heat maps and key
eye tracking metrics (KPIs).