Scribd
Upload
15 views1 page

Inner Speech Decoding for CLIS Patients

The document presents a machine learning-based system for decoding inner speech in Completely Locked-in Syndrome (CLIS) patients, utilizing Classical Machine Learning (CLM), 1D CNN, and 2D CNN models. The study involved preprocessing EEG data from two datasets, achieving the highest accuracy of 74.38% with the CLM model, while the 2D CNN model performed poorly at 32.47%. The research aims to enhance communication for CLIS patients by developing a user-friendly interface based on the recognized inner speech commands.

Uploaded by

Badreddin
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
15 views1 page

Inner Speech Decoding for CLIS Patients

The document presents a machine learning-based system for decoding inner speech in Completely Locked-in Syndrome (CLIS) patients, utilizing Classical Machine Learning (CLM), 1D CNN, and 2D CNN models. The study involved preprocessing EEG data from two datasets, achieving the highest accuracy of 74.38% with the CLM model, while the 2D CNN model performed poorly at 32.47%. The research aims to enhance communication for CLIS patients by developing a user-friendly interface based on the recognized inner speech commands.

Uploaded by

Badreddin
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

eMindReader: A Machine Learning-Based Decoding System for

Recognizing Inner Speech in Completely Locked-in Syndrome Patients


Department of Electrical Engineering
Diala Bushnaq, Malek Chabbouh, Raghad Aljindi
Dr. Muhammad Chowdhury, Dr. Muhammad Khan

ABSTRACT PREPROCESSING
This project proposes an Inner Speech decoding system for Completely Locked-in Syndrome The EEG recordings underwent preprocessing, including filtering from 0.5 to 64 Hz and
(CLIS) patients. The proposed design involves developing three models: Classical Machine baseline drift correction through polynomial fitting subtraction. Motion artifact removal utilized
Learning (CLM), One-Dimensional Convolutional Neural Network (1D CNN), and Two-Dimensional Variational Mode Decomposition. Subsequently, top PCA components' weights were observed
Convolutional Neural Network (2D CNN). Experimentation is done with 2 Inner Speech to select the most representative channels.
Electroencephalography (EEG) datasets: A publicly available Spanish dataset, and a collected
Arabic dataset. The dataset was collected from 5 subjects while abiding to specific standards
and protocols. Experimentation involves signal preprocessing, and classification which is done
over the 3 main subsystems. Results show variability in performance, where Classical Machine
Learning Model ETC achieved the highest accuracy and F1 scores of 74.38% and 74.18%
respectively, and other models yielded lower scores such as 30.29 for 1D CNN, 40.07% for
Self_ONN 1D, and an average 32.47% for 2D CNN

PROBLEM STATEMENT
Thousands of LIS and CLIS sufferers globally struggle to communicate their basic needs.
Developing a decoding system using CLM and DL for inner speech recognition could greatly
enhance their quality of life. significantly improving their quality of life. However, there are
certain obstacles when it comes to implementing CLM and DL for EEG signals, such as subject
variability, data scarcity, and noise interference. With the problem defined, the aims and
objectives are well understood.

METHODOLOGY IMPLEMENTATION AND RESULTS


In Classical Machine Learning, Feature Extraction is done over three domains: time, frequency, and
Initially, the collaborative work involves preprocessing the publicly available dataset “Thinking time-frequency. In total, there are 191 features extracted from each EEG channel. Subsequently,
Out Loud” [1]. This includes tasks such as filtering, epoching, baseline correction, and MA the best 30 features are selected using mRMR, and are used for training and testing the models.
removal to ensure a good quality signal. Following this stage, the project diverges into Evaluation is done by using 5-Fold Cross Validation with assessing accuracy and F1 score metrics.
individual pathways, where each student focuses on their unique subsystem: Classical Machine
Learning, 1D-CNN, and 2D-CNN.
After comparing each model’s performance, the best model will be chosen to classify the new
dataset, which is being collected locally from native Arabic speakers.

Among the models evaluated, ETC (n=500) achieved the highest accuracy and F1 scores of 74.38%
and 74.18% respectively. Below is a sample confusion matrix for subject 1 with ETC (n = 500).

Additionally, the SelectKBest function was used to


DATA COLLECTION optimize the selected features count, while
minimizing computational resources. It showed that
This dataset comprises six sessions for each of the five subjects: four inner 24 features correspond to a 2% decrease in
speech sessions, and two pronounced speech sessions in between. Six inner accuracy and F1 scores.
speech words/classes are to be analyzed: ‘up’, ‘down,’ ‘left’, ‘right’, ‘select’,
and ‘cancel’. The chosen words are intended for integration into a GUI to
enable patients to access a wider range of needs that would be displayed
DISCUSSION AND CONCLUSION
and selected using those 6 words. The figure on the right illustrates a screen
where the patient will have options to scroll through using his inner speech. Three models (classical machine learning, 1D-CNN, 2D-CNN) were developed to decode inner
speech from EEG signals using a public dataset. Preprocessing steps included filtering,
segmentation, and artifact removal.
Classical machine learning extracted 191 features, with mRMR feature selection preserving the
best 30. ETC algorithm achieved the highest accuracy of 72.07% and 74.38% for two datasets.
1D CNN showed a trade-off between performance and computational efficiency, with Self-
ONN outperforming it.
2D CNN, while showing poor performance (32.47% accuracy), suggests room for improvement
with different methods and architectures.

REFERENCES
The data acquisition process is done in 6 sessions, with breaks in between for subjects to
adjust their comfort, in order to maintain a balance between obtaining sufficient EEG data [1] N. Nieto, V. Peterson, H. Rufiner, J. Kamienkowski and R. Spies, “Thinking out loud, an open-
quantity and quality. Each trial in the recording protocol includes intervals for concentration, access EEG-based BCI dataset for inner speech recognition,” Scientific Data, vol. 9, p. 52,
cues, action, relaxation, and rest, with participants instructed to minimize movement during the 2022.
process to prevent motion artifacts.

Common questions

Powered by AI

Preprocessing EEG signals involved filtering frequency bands between 0.5 to 64 Hz, baseline drift correction using polynomial fitting subtraction, and motion artifact removal employing Variational Mode Decomposition. Additionally, Principal Component Analysis (PCA) helped in selecting the most representative channels. These steps are crucial to ensure the clarity and accuracy of the extracted signals, reduce noise and interference, and enhance the feature extraction process, which is vital for reliable machine learning models' performance .

The multi-session EEG data collection protocol is significant as it helps balance data quantity with quality, crucial for robust model training and testing. It involves several sessions with breaks for subject adjustment, which mitigates fatigue and ensures consistent signal quality. This protocol allows for capturing diverse EEG patterns and enhances the model's ability to generalize across varying states of concentration and relaxation, which is fundamental for developing reliable inner speech recognition systems .

The performance results indicate that the Classical Machine Learning Model ETC achieved the highest accuracy and F1 scores of 74.38% and 74.18% respectively, suggesting better applicability for practical use in inner speech recognition. In contrast, models like 1D CNN and 2D CNN showed lower performance with accuracies of 30.29% and 32.47%, implying that they might not be as reliable or ready for practical deployment without further optimization or redesign. The relative efficiency of the ETC model demonstrates potential feasibility in real-world applications, contingent on overcoming preprocessing challenges and feature extraction optimizations .

Filtering in EEG preprocessing targets specific frequency bands (0.5 to 64 Hz) to remove unwanted noise and signal artifacts that can distort EEG data. It addresses issues such as removing low frequency drifts and high frequency noise, ensuring that the data accurately represent the brain activity related to inner speech. Filtering improves the signal quality and is crucial for reliable feature extraction and subsequent model performance .

Classical and deep learning models play a crucial role in improving communication for CLIS sufferers by decoding EEG signals indicative of inner speech. These models can classify distinct word representations used in a GUI, potentially allowing patients greater interaction capabilities. However, limitations include variability in model performance, with classical models like ETC showing better accuracy than deep learning models, which require more optimization. Additional challenges include managing noise, subject variability, and data scarcity, which can compromise the models' reliability and accuracy .

Developing a decoding system for inner speech recognition in Locked-in Syndrome patients faces several challenges, including subject variability, which can lead to differing results across individuals; data scarcity, as acquiring sufficient high-quality EEG data is difficult; and noise interference, which can skew EEG signals and affect their interpretation. These obstacles complicate the reliable and accurate recognition of inner speech through Classical Machine Learning (CLM) and Deep Learning (DL) techniques .

The selected EEG signal class labels for CLIS patient communication included 'up', 'down', 'left', 'right', 'select', and 'cancel'. These were intended for integration into a GUI, allowing patients to navigate through and select different options using their inner speech. The aim was to enable a broader range of interactions and fulfill basic communication needs using these six elementary commands .

Subject variability affects the efficacy of EEG-based decoding systems in recognizing inner speech by introducing differences in EEG signals based on individual neurological and physiological factors. These discrepancies can lead to model inconsistencies and lower accuracy, as models may not generalize well across different subjects. This variability underscores the need for adaptive models and robust preprocessing protocols to manage inter-subject differences effectively .

Feature extraction in the time, frequency, and time-frequency domains impacts the model's overall results by providing a multidimensional representation of the EEG data, essential for capturing different signal characteristics. Time domain features may reflect amplitude changes, frequency domain features highlight power spectrum variations, and time-frequency features combine both aspects to track dynamic changes over time. These comprehensive features improve model training by offering a robust data set that leads to higher accuracy and reliability—as seen with the ETC model's performance—while reducing overfitting and improving generalization across datasets .

The mRMR (Minimum Redundancy Maximum Relevance) feature selection technique is used to identify the most informative features while reducing redundancy among them. By selecting the best 30 features from a total of 191 extracted from the EEG data, mRMR enhances the model's performance by ensuring that the chosen features contribute significantly to prediction accuracy. It optimizes computational efficiency and model generalization, leading to higher accuracy and F1 scores, as observed with the ETC model achieving 74.38% accuracy .

You might also like