Module Component Description
M1 Facial Emotion Classifier Upgraded CNN trained on AffectNet/RAF-DB
with transfer learning from a pre-trained
backbone (e.g., ResNet-50). Outputs 7-class
emotion probability vector.
M2 Text-Based Risk Classifier Fine-tuned BERT/RoBERTa model trained on
the Suicide Detection Dataset (Kaggle).
Classifies text input as 'suicide' or 'non-suicide'
with confidence score.
M3 Multimodal Fusion Layer Late-fusion module combining facial and text
embeddings using a learned weighted
combination. Outputs a unified risk score.
M4 Risk Assessment UI PyQt5 desktop application (building on existing
depression_classifier_pyqt5.py) that visualises
emotion probabilities, text risk score, and final
depression/risk status in real time.
2.1 Core Hypotheses
• H1 — Facial expression analysis alone is insufficient for reliable depression/suicide risk
classification due to individual variation in emotional expression.
• H2 — Text-based NLP models applied to self-reported language can identify linguistic
markers of suicidal ideation with higher specificity than facial classifiers alone.
• H3 — A late-fusion multimodal model combining facial and linguistic features will
outperform both unimodal baselines on F1-score and AUC metrics.
• H4 — The system can be deployed as a real-time, privacy-respecting tool that remains
clinically useful without replacing professional diagnosis.
3.4 Datasets
• FER-2013 (facial expressions, 35,887 images) — baseline continuity with prior work.
• AffectNet (450,000+ facial images with valence/arousal labels) — for improved facial
model generalization.
• Suicide & Depression Detection Dataset (Kaggle, ~232,000 Reddit posts) — for NLP
model training.
• DAIC-WOZ (Depression interviews, audio+video+text) — for multimodal fusion
evaluation.
3.5 Methodology
The project will follow an iterative, agile-inspired development cycle aligned with the following
phases:
1. Literature Review & Baseline Reproduction: Review current multimodal affective
computing research. Reproduce and document the original FER-2013 CNN baseline.
2. Facial Model Upgrade: Fine-tune a ResNet-50 backbone on AffectNet/RAF-DB.
Evaluate against the original FER-2013 model using accuracy, F1-score, and confusion
matrix.
3. NLP Model Training: Fine-tune a pre-trained transformer (BERT/RoBERTa) on the
Suicide Detection Dataset. Evaluate with precision, recall, AUC-ROC.
4. Multimodal Fusion: Design and evaluate a late-fusion architecture. Compare unimodal
vs. multimodal performance.
5. Application Integration: Extend the existing PyQt5 desktop application to incorporate text
input, real-time webcam feed, and a unified risk dashboard.
6. Validation & Ethical Review: Conduct user testing (with synthetic/consent-given data)
and document ethical safeguards.
3.7 Ethical Considerations
Given the sensitive nature of suicide and depression screening, the project will adhere to the
following ethical principles:
• Informed Consent: All data used for training will be publicly available or ethically
sourced. No live clinical data will be collected without explicit institutional approval.
• Data Privacy: All personal data processed by the application will remain local and will not
be transmitted to external servers.
• Non-Diagnostic Framing: The system is explicitly designed as a pre-screening aid, not a
diagnostic tool. All outputs will include clinical disclaimer language.
• Bias Auditing: The model will be tested across demographic subgroups (age, gender,
ethnicity) to identify and mitigate classification bias.
• GDPR Compliance: The system will be designed in compliance with applicable data
protection regulations.
3.9 Resources Required
• Hardware: GPU-enabled workstation or cloud compute (Google Colab Pro / AWS EC2
GPU instance) for model training.
• Software: Python 3.10+, TensorFlow/Keras, HuggingFace Transformers, OpenCV,
PyQt5, pandas, scikit-learn.
• Datasets: FER-2013, AffectNet (license required), Suicide Detection Dataset (Kaggle),
DAIC-WOZ (agreement required).
• Infrastructure: GitHub for version control, Weights & Biases for experiment tracking,
LaTeX/Word for documentation.