2023 IEEE International Conference On Metrology for eXtended Reality, Artificial Intelligence, and Neural Engineering (MetroXRAINE)

A Brain-Computer Interface Augmented Reality

Framework with Auto-Adaptive SSVEP Recognition
Yasmine Mustafa∗§ , Mohamed Elmahallawy∗ , Tie Luo∗ , Seif Eldawlatly †‡
∗ Computer Science Department, Missouri University of Science and Technology, MO, USA
† Computer and Systems Engineering Department, Ain Shams University, Egypt
‡ Computer Science and Engineering Department, The American University in Cairo, Egypt
§ Digital Media Engineering and Technology Department, German University in Cairo, Egypt
Emails: {yam64, tluo, meqxk}@[Link], seldawlatly@[Link]

Abstract—Brain-Computer Interface (BCI) initially gained at- between being immersed in the application and being present in
arXiv:2308.06401v1 [[Link]] 11 Aug 2023

tention for developing applications that aid physically impaired the real world.
individuals. Recently, the idea of integrating BCI with Augmented The idea of merging BCI with Extended Reality (XR), such
Reality (AR) emerged, which uses BCI not only to enhance the
quality of life for individuals with disabilities but also to develop as AR and VR, was introduced by Jantz et al. [4] to address the
mainstream applications for healthy users. One commonly used limitations of XR controls such as hand gestures and vocal com-
BCI signal pattern is the Steady-state Visually-evoked Potential mands. For example, hand gestures break the user’s immersion
(SSVEP), which captures the brain’s response to flickering visual in AR and VR, limit users to objects at arm length, and have
stimuli. SSVEP-based BCI-AR applications enable users to express safety concerns of hitting objects in the physical world. Voice
their needs/wants by simply looking at corresponding command
options. However, individuals are different in brain signals and controls function poorly in loud environments and can be socially
thus require per-subject SSVEP recognition. Moreover, muscle embarrassing. Thus, the integration of BCI and AR opens up new
movements and eye blinks interfere with brain signals, and thus opportunities in gaming, enterprise, and medical applications.
subjects are required to remain still during BCI experiments, Hence, integrating BCI (instead of external controls) and AR as
which limits AR engagement. In this paper, we (1) propose a an emerging paradigm is more promising which both enhances
simple adaptive ensemble classification system that handles the
inter-subject variability, (2) present a simple BCI-AR framework user experience and opens up new application opportunities.
that supports the development of a wide range of SSVEP-based Moreover, combining BCI and AR
BCI-AR applications, and (3) evaluate the performance of our The most popular BCI is the electroencephalogram (EEG)
ensemble algorithm in an SSVEP-based BCI-AR application with based BCI because of its safety and reasonable temporal res-
head rotations which has demonstrated robustness to the movement olution. One of the control signals used in BCI based on
interference. Our testing on multiple subjects achieved a mean
accuracy of 80% on a PC and 77% using the HoloLens AR headset,
EEG systems is the Steady-State Visually-Evoked Potentials
both of which surpass previous studies that incorporate individual (SSVEPs). SSVEP is a resonance phenomenon that can be
classifiers and head movements. In addition, our visual stimula- observed through electrodes placed on the scalp (specifically over
tion time is 5 seconds which is relatively short. The statistically the occipital and parietal lobes of the brain) when a subject looks
significant results show that our ensemble classification approach at a light source (stimulus) flickering at a constant frequency
outperforms individual classifiers in SSVEP-based BCIs.
[5]. When a person gazes at a specific stimulus, SSVEP signals
I. I NTRODUCTION are generated in the brain as a natural response to the stimulus
with the same frequency or multiples of the frequency of that
Brain-computer Interface (BCI) enables the human brain to stimulus. In EEG recordings, the spectral power of the SSVEP
interact with an external device by sending and receiving just responses is compared and analyzed for different purposes. One
brain signals, allowing people to control computers or other popular usage is to relate each EEG tag to a specific control
devices with no stimulation of muscles. It uses electrodes to command to design an application, where such an SSVEP-based
capture brain activity and translates it to commands that can BCI application allows the user to access a variety of control
be sent to external devices to take desired actions [1]. In the commands simply by focusing her attention on them.
past, investigations in the field of brain activity were limited One of the first studies that introduced BCI-AR into main-
to medical cases and studying neurological disorders in clinics, stream applications is Si-Mohammed et al. [6]. They focused
but recently, computing and biosensing innovations have signif- on examining how head movements and how the layout of
icantly improved the outlook for BCI applications, and many the designed visual stimuli affect performance. In the online
studies have appeared that create assistive tools for people with experiments, mean accuracy, when head movements were incor-
disability to regain control and communication abilities [1], such porated, dropped from 78% to 41%, indicating the need for more
as controlling a prosthetic arm [2]. robust systems. In addition, the experiments also showed that the
More recently, BCI has started to evolve only as an aid performance depended on the match between the layout design
for physically impaired individuals to games and mainstream and user preference, indicating the issue of subject dependency
applications for normal people [3]. One direction for developing when it comes to brain signals.
mainstream BCI applications is integrating BCI with Augmented Addressing subject-dependent brain signals, Yue et al. devel-
Reality (AR). Unlike Virtual Reality (VR) which completely oped an attention-aware system [7] which tries to limit the virtual
isolates the user from the actual environment, AR provides content displayed to a user in order to reduce distractions caused
an environment that shows both the virtual contents and the by surroundings to her, thereby improving the performance of
actual environment. By doing so, it offers users a middle ground SSVEP recognition by aligning better with the user’s brain

1
signals. Although considering the attentional state can serve as
a useful guideline for adjusting visual content, it alone is insuf-
ficient to address all the complexities of subject-dependent brain
signals. In fact, relying solely on attentional state information
yielded a mean accuracy of only 65.6% [7].
A more recent study was conducted by Zhang et al., where
they focused on controlling a robot’s movement in a maze [8].
Zhang et al. used AR as a portable visual stimulation device,
which means that the user does not need to switch their focus
back and forth between the environment and the stimulation
device. Results showed that all subjects successfully completed
the robot walking task, demonstrating the feasibility of the AR-
Fig. 1: Overview of the full pipeline between BCI implemented in
SSVEP-NAO system. Ke et al. also showed promising results in Python and AR implemented in Unity.
controlling a robotic arm in a high-speed online SSVEP-based
BCI-AR environment and discussed the limitations that can be
imposed by head-mounted displays such as the HoloLens as they Epoc EEG headset. The software implementation involves an AR
can provide less stable frame rates than the computer screens [9]. interface, EEG signal recording, preprocessing, classification,
The mentioned work shows the feasibility of integrating AR and the interface between the AR and BCI components. Fig. 1
and BCI and encourages the development of more applications shows an overview of the framework where the HoloLens AR
and ideas for hands-free and voice-free user interfaces [7]. These headset displays buttons as visual stimuli, while the Emotiv head-
studies inspired us to extend the exploration of AR-BCI environ- set records and transmits EEG signals for processing in Python.
ments and overcome common design challenges: 1) the artifacts The recorded data is then read, preprocessed, and classified using
that can be introduced to the EEG signal through blinking eyes our ensemble machine-learning model. The classification results
and muscle movements require the user to stay still throughout are communicated from Python to Unity via a Python-Unity-
the entire experiment, and 2) the uniqueness of the EEG signal Socket [10]. Unity, in turn, presents relevant feedback to the
of each individual creates the variation of performance among user on the HoloLens in response to each input.
subjects. Meeting these challenges necessitates an approach that
can handle unique brain signals, while also being robust enough B. Application
to accommodate essential movements in an AR environment,
like head rotations, that introduce artifacts during the process. The application implemented in this work is developed using
In light of these challenges, this paper makes the following Unity on top of an architecture of an open-source HoloLens
contributions: project [11]. This architecture originally had three vocal com-
mands that correspond to three actions: Create Cube, Create
• We propose a BCI-AR framework that is robust to artifacts
Sphere, and Delete All.
and easy to implement for creating SSVEP-based BCI-AR
Since we want to prove that we can replace the vocal com-
applications.
mands with SSVEP-based BCI input, we added three flickering
• To allow an immersive AR user experience, we allow head
buttons (visual stimulation) to the scene that correspond to the
movements which was difficult in previous studies due to the
same three commands. If Create Cube button is selected (via
sensitivity of brain signals to various factors such as external
the SSVEP-based BCI), a cube is created in space where the
distractions, user’s inattention and muscle movements.
cursor of the HoloLens is pointed at. Similarly, selecting the
• We utilize an ensemble SSVEP recognition system that auto-
Create Sphere button results in the creation of a sphere at the
adapts to different subjects and movements and show that it
cursor location. Lastly, choosing the Delete All button removes
is better than individual classification models.
any previously created shapes. Fig. 2 demonstrates the design of
• Our experiments of using the HoloLens and PC with BCI
the BCI-AR application in which the three green buttons are the
demonstrate that the HoloLens provides accuracy close to
flickering buttons that represent the visual stimulation, and the
that provided by the PC, which means that the HoloLens is
shapes created in space are examples of the feedback provided
on par with the PC, the conventional device used with BCI.
to the user upon her/his button gaze selection.
Our experiments show promising results that are superior to
previous studies that allowed head movements in BCI-AR C. SSVEP Interface Specifications
systems, as well as better accuracy across different subjects,
indicating good generalizability. Flickering frequencies and colors of the buttons are selected
based on the study done by Duart et al. [12]. The frequencies
II. F RAMEWORK assigned to each button are as follows: Create Cube flickers at
In this section, we begin by presenting an overview of the a frequency of 12 Hz, Create Sphere at 8.57 Hz, and Delete
proposed system, followed by a description of its application. All at 10 Hz. Duart et al. studied the effect of colors on the
Then we highlight the key specifications of the SSVEP interface signal-to-noise ratio (SNR) of the EEG signal. They used the red,
to provide a better understanding of its importance. white, and green colors and frequencies 5 Hz (low frequency),
12 Hz (medium frequency), and 30 Hz (high frequency) in their
A. System Overview study. For low frequencies, the green color was found to show
The proposed system comprises two essential hardware com- the best results. In our system, given that we use low flickering
ponents: the first-generation Microsoft HoloLens and the Emotiv frequencies, the green color was selected for the buttons.
 periods and records only during the flickering part of the trial.
For each trial recorded, there are 14 columns that correspond to
the 14 channels and 1285 rows for each channel. The sampling
rate is 257 Hz.
2) Preprocessing and SSVEP Recognition: First, common
average reference (CAR) filter is applied. The CAR filter is
a spatial filter that removes the averaged brain activity across
all electrodes for a specific time instant. This helps to remove
the common features across all electrodes, hence reducing noise
[13]. The formula used to compute the CAR filter takes the
following form
CAR ER 1
Pn ER
Ui (t) = Ui (t) − n j=1 Uj (t) (1)
Fig. 2: Our application deployed on the HoLoLens. The three green
buttons correspond to 3 different commands to be selected using BCI where Ui (t)
ER
is the potential difference between the ith elec-
input.
trode and the ear reference, and n is the number of electrodes
in the headset. For every electrode i, the average is subtracted
CAR
from the potential difference, resulting in Ui (t) .
Next, Fast Fourier Transform (FFT) is applied to the signal
recorded on each channel (data recorded from an electrode).
Next, only the flickering frequencies and their harmonics are
extracted from the resulting power spectrum. The peak is not
Fig. 3: Subjects wearing the Emotiv Headset and the HoloLens. necessarily visible at the frequency itself. This may be due to
the head rotations and eye blinking as muscle movements are
known in the literature to introduce artifacts in the signal [14],
III. M ETHODS but it’s more likely to be due to the unstable frame rates in
In this section, we provide comprehensive descriptions of both Microsoft Hololens [15]. To compensate for this shift in peak
the hardware and software components used in this study. The frequencies, a window of size 1 Hz, ranging from 0.5 Hz before
software encompasses various aspects, including data specifica- the frequency to 0.5 Hz after the frequency, is taken at each
tions, preprocessing techniques, and classification methods. frequency and its harmonics.
Next, Principal Component Analysis (PCA) is applied to
A. Hardware compress important features into fewer columns and decrease
For EEG recording, we used the Emotiv Epoc+, a 14- the running time [16]. Data is then normalized using the Z-score
channel EEG headset with electrodes positioned at AF 3, method [17].
F 7, F 3, F C5, T 7, P 7, O1, O2, P 8, T 8, F C6, F 4, F 8, andAF 4. Online experiments are done using channels O1 and O2 only
These electrodes are saline-based and do not require the use of as these are the electrodes covering the visual cortex area where
sticky gel, ensuring a more comfortable experience for the user. SSVEP is most prominent. However, to analyze and compare the
In the context of AR, we utilized the first generation of results, preprocessing is done offline using the concatenation of
Microsoft HoloLens. This AR device enables the display of 3D all the channels first.
content seamlessly integrated with the real world surrounding 3) Classification Techniques: Mainly the two machine learn-
the user. To enhance user comfort, the HoloLens is equipped ing classifiers used in the experiments are Support Vector Ma-
with nose pads, overhead straps that reduce its weight burden, chine (SVM) and Random Forests (RF). SVM has been widely
and a rotating headband that can be adjusted to fit the shape and used by researchers to classify EEG data. In previous studies,
size of each user. For visual reference, Figure 3 showcases the it showed better performance than neural networks [18]. SVM
participation of different subjects in this study, all wearing both performed best in this work with linear and polynomial kernels.
headsets simultaneously. RF as well showed promising results in recent studies that
classified EEG data [19].
B. Software The classification of the offline experiments involved splitting
1) Data Recording Specifications: Each of the three buttons in the recorded data into two sets: 80% for training and 20% for
the application flicker with its specified frequency for 5 seconds testing, ensuring that each set maintained equal proportions of
followed by a resting period of 5 seconds, creating what is each class. As our approach is subject-based, we implemented
referred to as a trial that has a total duration of 10 seconds. A subject-wise stratified partitioning. Subject-wise stratified parti-
session is the term used when a number of trials are conducted tioning ensures that the data from each individual subject are
consecutively. For each subject, 5 sessions are conducted. Each kept together while maintaining data of each class within the
session is divided into two parts: Session a contains 12 trials, and training and testing sets, allowing the model to learn subject-
session b contains 13 trials, creating a total of 25 trials for each specific patterns.
session. There is a 30-second resting period between sessions a The ensemble model consists of four varieties of each classi-
and b. A synchronization clock organizes the start and end of fier: PCA and CAR filter, CAR filter only, PCA only, and without
both the data recording of the Emotiv headset and the flickering PCA or CAR filter. This results in eight variations: four for the
of the buttons such that the Emotiv does not record the resting SVM classifier and four for the RF classifier.
 26 years. In order to ensure a fair and unbiased assessment,
there was no preference given to specific genders or hair types
(in terms of thickness or length), aiming to maintain an equitable
mean accuracy. Volunteers from all backgrounds were welcomed
and actively encouraged to participate in the study.
Offline experiments were used to describe the data analysis
conducted after recording the data, where no real-time feedback
is provided to the user. The experimental sessions consisted
of five individual sessions, each further divided into session a
Fig. 4: Procedure of the online preprocessing and classification per trial. and session b. In session a, the user selects buttons based on
a predetermined sequence, and a different sequence is used in
session b. For instance, a sequence can be 2, 0, 1, 2, 0, 2, 1, 0,
For every trial, the predictions of all models are considered and 1, 0, 1, 2, where 0 corresponds to "create cube", 1 corresponds
the label with the highest weighted number of votes is the output to "delete all", and 2 corresponds to "create sphere". The user is
of the ensemble model. The vote of every label is multiplied required to direct their gaze to the respective buttons as indicated
by the training accuracy of the model that predicted that label, by the appearance of a red dot during each trial. A resting period
creating a weighted ensemble classifier [20]. Weighted votes give of 5 seconds is provided between trials to allow the user to shift
more weight to classifiers that are more likely to give better their focus to the next button. Following the offline experiments,
performance and less weight to classifiers that are less likely the online experiments involve providing feedback to the user
to give good results. Thus, classifiers with worse performance based on their gaze selection, while also allowing them to freely
are prevented from decreasing the overall ensemble accuracy. move their head to navigate the environment and determine the
The final accuracy is calculated based on the output that got a desired location for creating shapes using the head cursor. The
maximum number of votes. Fig. 4 gives an overview of the online classification model used in the online experiments is pre-trained
system. CAR filter and FFT both demonstrated high performance during the offline experiments.
compared to other preprocessing methods in the literature [21,
22]. A. Experiments using Personal Computer (PC)
1) Offline Experiments: In order to examine the performance
C. Evaluation Metrics of the proposed system before integrating it with the Hololens
The performance of most BCI systems is evaluated using AR set, we sought to design an experiment that runs on a PC
accuracy and Information Transfer Rate (ITR) [23]. Accuracy first. The performance of using all channels in the analysis is
is the percentage of correctness in predicting the output of the examined in an offline setup in which subjects were instructed
user gaze. A correct output happens when the output label is that to focus on specific buttons (targets) in a pre-specified sequence.
of the command the user gazed at. Accuracy is defined as the Fig. 5 shows the accuracy obtained for each subject using the PC.
total number of correct predictions divided by the total number The figure compares the performance for different pre-processing
of predictions. On the other hand, ITR does not only consider the stages (No CAR or PCA, using PCA only, using CAR only, and
number of correct predictions but also relates the accuracy with using CAR and PCA) and different classifiers (SVM and RF).
the number of classes and stimulation time. ITR can be defined The mean accuracy using all channels for the PC experiments
as follows where B is the information transferred in bits per reached 80%.
trial, N is the number of commands, and P is the classification Given that channels O1 and O2 are located on the visual
accuracy [24]. cortex that is responsible for processing visual inputs [25], we

Bit
 
1−P
 next examined the performance when signals from channels
B T rial = log2 N + P × log2 P + (1 − P ) × log2 N −1 (2) O1 and O2 only were used as opposed to using all channels.
To calculate the information transferred in bits per minute, the Fig. 6 demonstrates the accuracy achieved for the same subjects
result of the above equation, B, is multiplied by the average using different pre-processing stages and classifiers. The figure
classification time in minutes, Q [24]. demonstrates an enhancement in the maximum classification
  accuracy for subjects S1 (69.33% and 60.68 bits/min) and S2
IT R Bit
=B×Q (3) (79.47% and 100.98 bits/min) compared to using all channels in
M in
the analysis, while subject S3’s maximum classification accuracy
remained unchanged (80% and 103 bits/min). This confirms our
IV. R ESULTS hypothesis that using O1 and O2 only would have a positive
We consider four different experiments to evaluate our pro- impact on the performance.
posed method: i) offline experiments on the PC, ii) offline experi- 2) Online Experiments: We next examined the feasibility of
ments on the HoloLens, iii) online experiments on the PC, and iv) using the proposed approach in an online setup. The subject is
online experiments on the HoloLens. Multiple experiments with then allowed to freely focus on any of the targets while the
different setups (design and layout) were performed to identify system attempts to identify the target intended. Given that the
the setup that gives the best online performance. A total of six offline analysis indicates that using O1 and O2 only enhances
volunteers took part in the experiments, with three assigned to the performance, in the online analysis, only signals from these
the PC experiments (Subjects S1, S2, and S3) and the remaining two channels were considered. However, as Fig. 6 demonstrates
three assigned to the HoloLens experiments (Subjects S4, S5, for the offline analysis, we faced a problem that there is not a
and S6). The age range of the participants was between 18 and single pre-processing or classification technique that outperforms
(a) Subject S1 accuracy. (b) Subject S2 accuracy. (c) Subject S3 accuracy. (a) Subject S4 accuracy. (b) Subject S5 accuracy. (c) Subject S6 accuracy.

(d) Subject S1 ITR. (e) Subject S2 ITR. (f) Subject S3 ITR. (d) Subject S4 ITR. (e) Subject S5 ITR. (f) Subject S6 ITR.

Fig. 5: Accuracy for offline experiments on all channels and PC. Fig. 7: Accuracy for offline experiments using O1 and O2 channels and
Hololens.

TABLE II: Accuracy in online experiments by the Ensemble classifier

using channels O1 and O2 on Hololens.
Subject Online Accuracy (%) ITR (bpm)
S4 75.17 82.38
S5 77.76 93.29
S6 75.58 76.39
(a) Subject S1 accuracy. (b) Subject S2 accuracy. (c) Subject S3 accuracy.

from channels O1 and O2 only given the enhancement achieved

using these channels in the PC experiments. We first examined
the performance in an offline setup. Fig. 7 demonstrates the
accuracy obtained for each of the subjects when testing on the
HoloLens for different pre-processing stages and classifiers. The
(d) Subject S1 ITR. (e) Subject S2 ITR. (f) Subject S3 ITR. mean accuracy achieved across all three subjects using O1 and
O2 channels on HoloLens is 73.39%. Comparing these results to
Fig. 6: Accuracy for offline experiments using O1 and O2 channels and
PC.
what was achieved using the PC indicates that using HoloLens is
just as effective as using a PC as there is no significant difference
between the accuracy of the HoloLens and the PC.
all other techniques for all subjects. For Subject S1, the highest 2) Online Experiments: Consistent with the PC experiments,
accuracy 69.33% was obtained using CAR filter pre-processing the results in Fig. 7 emphasize that no certain combination
and RF classifier. For Subject S3, the highest accuracy 80.0% of pre-processing and a classifier gives the best performance.
was obtained using CAR pre-processing and SVM classifier. This For example, For Subject S4, the highest accuracy of 78.15%
problem motivated the examination of the ensemble classification (95.01 bits/min) was obtained using CAR filter pre-processing
technique as it could improve the classification accuracy by and Random Forests classifier. For Subject S5, the highest
voting across all pre-processing and classification techniques. accuracy of 76.45% (87.67 bits/min) was obtained using PCA
Table I summarizes the ensemble accuracy results obtained using pre-processing and Random Forests classifier. For Subject S6,
the PC. The accuracies achieved are within the range of the the highest accuracy of 65.57% (48.64 bits/min) was obtained
accuracies achieved in the offline analysis, indicating that there using CAR filter pre-processing and SVM classifier. Therefore,
is no overfitting. the ensemble classification technique is used for the online
experiments. Table II summarizes the accuracies achieved in
B. Experiments using HoloLens the HoloLens online Experiments. The results indicate a mean
1) Offline Experiments: We next examined the integrated accuracy across all subjects of 76.2%. To assess the significance
system that uses the Hololens AR set, which is the main objective of the results obtained from the ensemble classifier in comparison
of this work. In these experiments, we used signals recorded to the best-performing individual classifiers, a paired t-test was
conducted. The analysis revealed a statistically significant differ-
ence, with a p-value of 0.00855, indicating that the performance
TABLE I: Accuracy in online experiments by the Ensemble classifier of the ensemble classifier outperformed that of the individual
using channels O1 and O2 on PC.
classifiers.
Subject Online Accuracy (%) ITR (bpm)
S1 77.29 91.28 V. C ONCLUSION
S2 72.22 70.94 We present a proof-of-concept study that shows the feasibility
S3 81.32 109.73 of using SSVEP-based BCI input with allowed head rotations to
navigate through the AR environment without conventional BCI- [12] X. Duart, E. Quiles, F. Suay, N. Chio, E. García, and
imposed restrictions such as staying still. Head movements are F. Morant, “Evaluating the effect of stimuli color and
not only allowed between SSVEP commands but also during the frequency on ssvep,” Sensors, vol. 21, no. 1, p. 117, 2021.
SSVEP commands. No significant influence on the results was [13] O. Bertrand, F. Perrin, and J. Pernier, “A theoretical jus-
observed despite the movements, and the results are comparable tification of the average reference in topographic evoked
to prior work such as [6, 26] whose performance dropped potential studies,” Electroencephalography and Clinical
significantly when movements occurred. In addition, the required Neurophysiology/Evoked Potentials Section, vol. 62, no. 6,
stimulation time of this study is only 5 seconds which is less than pp. 462–464, 1985.
the stimulation time needed by previous studies [6, 19, 26]. A [14] H. Bakardjian, T. Tanaka, and A. Cichocki, “Optimization
paired t-test revealed a significant difference between the results of ssvep brain responses with application to eight-command
obtained from our ensemble classifier and the best-performing brain–computer interface,” Neuroscience letters, vol. 469,
individual classifier. Online and offline experiments were done no. 1, pp. 34–38, 2010.
with subjects of different genders and hair types. Using channels [15] P. Arpaia, E. De Benedetto, L. De Paolis, G. D’Errico,
O1 and O2 only instead of using all 14 channels significantly N. Donato, and L. Duraccio, “Performance enhancement
increased the mean accuracy. Then, an ensemble classifier is of wearable instrumentation for ar-based ssvep bci,” Mea-
used to overcome the inconsistent performance among subjects. surement, vol. 196, p. 111188, 2022.
Moreover, our BCI-AR integration study shows that on a simple [16] M. K. M. Rabby et al., “Epileptic seizures classification
HoloLens, we can achieve accuracy comparable to on a PC and in eeg using pca based genetic algorithm through machine
provide seamless communication in an AR environment. learning,” in Proceedings of the 2021 ACM Southeast
Conference, 2021, pp. 17–24.
R EFERENCES [17] L. Al Shalabi and Z. Shaaban, “Normalization as a pre-
processing engine for data mining and the approach of
[1] D. Pal, S. Palit, and A. Dey, “Brain computer interface: A preference matrix,” in 2006 International conference on
review,” Computational Advancement in Communication, dependability of computer systems. IEEE, 2006.
Circuits and Systems, pp. 25–35, 2022. [18] V. Saccá et al., “On the classification of eeg signal by using
[2] D. J. McFarland and J. R. Wolpaw, “Brain-computer inter- an svm based algorithm,” in Multidisciplinary approaches
faces for communication and control,” Communications of to neural computing. Springer, 2018, pp. 271–278.
the ACM, vol. 54, no. 5, pp. 60–66, 2011. [19] L. Yue, H. Shen, S. Wang, R. Boots, G. Long, W. Chen,
[3] D. Tezza et al., “Brain-controlled drone racing game: and X. Zhao, “Exploring bci control in smart environments:
A qualitative analysis,” in International Conference on Intention recognition via eeg representation enhancement
Human-Computer Interaction. Springer, 2020. learning,” ACM Transactions on Knowledge Discovery from
[4] J. Jantz, A. Molnar, and R. Alcaide, “A brain-computer in- Data (TKDD), vol. 15, no. 5, pp. 1–20, 2021.
terface for extended reality interfaces,” in ACM SIGGRAPH [20] Y. M. Hussain and S. Eldawlatly, “An ensemble classifica-
2017 VR Village, 2017, pp. 1–2. tion approach for recognizing steady-state visually evoked
[5] Z. İşcan and V. V. Nikulin, “Steady state visual evoked potentials frequencies,” in 16th International Conference on
potential (ssvep) based brain-computer interface (bci) per- Computer Engineering and Systems. IEEE, 2021.
formance under different perturbations,” PloS one, vol. 13, [21] R. Zerafa, T. Camilleri, O. Falzon, and K. P. Camilleri,
no. 1, p. e0191673, 2018. “To train or not to train? a survey on training of feature
[6] H. Si-Mohammed et al., “Towards bci-based interfaces extraction methods for ssvep-based bcis,” Journal of Neural
for augmented reality: feasibility, design and evaluation,” Engineering, vol. 15, no. 5, p. 051001, 2018.
IEEE transactions on visualization and computer graphics, [22] M. Kołodziej, A. Majkowski, Ł. Oskwarek, and R. J. Rak,
vol. 26, no. 3, pp. 1608–1621, 2018. “Comparison of eeg signal preprocessing methods for ssvep
[7] L.-M. Vortmann and F. Putze, “Attention-aware brain com- recognition,” in 2016 39th International Conference on
puter interface to avoid distractions in augmented reality,” Telecommunications and Signal Processing (TSP). IEEE,
in Extended Abstracts of the 2020 CHI Conference on 2016, pp. 340–345.
Human Factors in Computing Systems, 2020, pp. 1–8. [23] C.-C. Hsu et al., “Extraction of high-frequency ssvep for
[8] S. Zhang, X. Gao, and X. Chen, “Humanoid robot walking bci control using iterative filtering based empirical mode
in maze controlled by ssvep-bci based on augmented reality decomposition,” Biomedical Signal Processing and Con-
stimulus,” Frontiers in Human Neuroscience, vol. 16, p. trol, vol. 61, p. 102022, 2020.
908050, 2022. [24] J. R. Wolpaw et al., “Brain–computer interfaces for com-
[9] Y. Ke, P. Liu, X. An, X. Song, and D. Ming, “An online munication and control,” Clinical neurophysiology, vol.
ssvep-bci system in an optical see-through augmented re- 113, no. 6, pp. 767–791, 2002.
ality environment,” Journal of neural engineering, vol. 17, [25] B. Chen et al., “A stimulus-response based eeg biometric
no. 1, p. 016066, 2020. using mallows distance,” CCF Transactions on Networking,
[10] Y. E. Ashry, “Python-Unity-Socket-Communication,” https: vol. 3, no. 2, pp. 128–139, 2020.
//[Link]/yc538hhc, 2020, [Online; accessed 17- [26] G. Milsap and P. Peranich, “A low-cost research platform
October-2021]. for brain-computer-interface applications in mixed reality,”
[11] A. Mancino, “HololensVocalCommands,” [Link] in 2021 10th International IEEE/EMBS Conference on
om/AmerigoM/HololensVocalCommands, 2020, [Online; Neural Engineering (NER). IEEE, 2021, pp. 515–518.
accessed 17-October-2021].