A CNN–TCN-Based Tiny Temporal Chronic diseases account for a significant
Transformer with Efficient Self-Attention for proportion of global mortality and healthcare
Embedded AI-Based Smart Wearable Chronic expenditure. Early detection and continuous
Disease Detection monitoring are essential to prevent disease
progression and enable timely intervention.
Neethu Sebastian
Conventional healthcare monitoring systems
Department of
are largely episodic and hospital-centric,
University / Institute Name
limiting their ability to capture early
Email: ………
physiological changes. In contrast, smart
wearable devices allow continuous acquisition
of biosignals such as electrocardiogram (ECG),
Abstract photoplethysmogram (PPG), respiration,
Chronic diseases such as cardiovascular temperature, and motion data.
disorders, hypertension, diabetes, and Recent advances in deep learning have
respiratory illnesses require continuous and demonstrated strong performance in
early-stage monitoring to reduce morbidity physiological signal analysis. However,
and healthcare burden. Smart wearable deploying such models on wearable platforms
devices equipped with embedded artificial is challenging due to constraints in
intelligence (AI) offer a promising solution by computation, memory, and battery life. While
enabling real-time physiological signal analysis CNNs and recurrent neural networks (RNNs)
directly on resource-constrained hardware. such as LSTMs have been widely used, they
However, achieving high diagnostic accuracy often struggle to capture long-term temporal
while maintaining low power consumption, dependencies efficiently. Transformers with
memory footprint, and inference latency self-attention address this limitation but are
remains a major challenge. This report typically too computationally expensive for
presents a CNN–TCN-based Tiny Temporal embedded deployment.
Transformer (TTT) architecture with efficient
self-attention, designed specifically for This report proposes a CNN–TCN-based Tiny
embedded wearable platforms. The proposed Temporal Transformer with efficient self-
model combines convolutional neural attention, designed to overcome these
networks (CNNs) for local feature extraction, limitations and enable accurate, low-power,
temporal convolutional networks (TCNs) for on-device chronic disease detection.
long-range temporal modeling, and
lightweight self-attention mechanisms for
adaptive focus on clinically relevant signal II. Related Work
segments. The architecture provides an
Early wearable health monitoring systems
optimal trade-off between accuracy, early
relied on threshold-based or statistical
disease sensitivity, and energy efficiency,
methods, which lack robustness under real-
making it well suited for real-time chronic
world noise and inter-subject variability.
disease risk assessment on wearable devices.
Machine learning models such as support
Keywords— Smart wearable, Embedded AI, vector machines and random forests improved
Chronic disease detection, Temporal performance but require manual feature
Transformer, Efficient attention, TinyML engineering.
Deep learning approaches, particularly 1D
CNNs, have achieved strong results in ECG and
I. Introduction
PPG classification tasks. LSTM-based models
introduced temporal modeling capabilities but making them suitable for wearable
incur high memory and latency costs. environments.
Recently, attention mechanisms and
C. Temporal Convolutional Network (TCN)
transformers have shown superior
performance in time-series analysis by TCNs are used to model longer temporal
modeling long-range dependencies. However, dependencies through dilated causal
standard transformers exhibit quadratic convolutions. Compared to RNNs, TCNs offer
complexity with respect to sequence length, parallel computation, lower latency, and
making them unsuitable for embedded stable gradients, which are advantageous for
wearables. embedded inference.
To address this gap, lightweight attention D. Tiny Temporal Transformer with Efficient
mechanisms and hybrid architectures have Self-Attention
been explored. The proposed CNN–TCN-based
Tiny Temporal Transformer builds upon these A lightweight transformer encoder is
advances by integrating efficient self-attention introduced after the CNN–TCN stages. To
into an embedded-friendly temporal modeling ensure embedded feasibility, the following
framework. design choices are adopted:
Reduced number of attention heads
(1–2 heads)
III. Proposed Architecture
Short sequence length
A. System Overview
Linear or kernel-based efficient
The proposed system processes multimodal attention mechanisms
physiological signals acquired from wearable
sensors and performs on-device inference to The self-attention module enables the model
estimate early chronic disease risk. The to emphasize clinically significant time
architecture is optimized for deployment on segments, improving early disease sensitivity.
microcontroller-based platforms. E. Output Layer
Processing Pipeline: The final layer produces a disease risk score or
1. Signal acquisition and preprocessing classification label, enabling early alerts and
continuous monitoring.
2. Local feature extraction using CNN
layers
3. Temporal dependency modeling using IV. Embedded Optimization Strategies
TCN layers To meet wearable constraints, several
4. Efficient self-attention for adaptive optimization techniques are applied:
temporal focus Quantization: INT8 quantization to
5. Risk classification or regression output reduce memory and power
consumption
B. CNN-Based Local Feature Extraction
Pruning: Removal of redundant
One-dimensional CNN layers are employed to weights to reduce model size
extract local morphological features from
physiological signals such as ECG waveforms Latency-aware design: Limiting
and PPG pulses. These layers are sequence length and attention
computationally efficient and robust to noise, complexity
Hardware acceleration: Use of ARM wearable chronic disease detection. The
CMSIS-NN and TensorFlow Lite Micro proposed architecture effectively captures
both local and long-range temporal patterns
These optimizations enable real-time
while remaining compatible with resource-
inference with low energy consumption.
constrained wearable platforms. By providing
a favorable trade-off between accuracy, early
disease sensitivity, and energy efficiency, the
V. Performance Evaluation model serves as a strong baseline for next-
The proposed model is evaluated using generation intelligent health monitoring
publicly available physiological datasets such systems.
as WESAD, PPG-DaLiA, and MIT-BIH ECG.
Performance metrics include accuracy,
sensitivity, inference latency, and memory References
footprint.
[1] A. Author et al., “Deep learning for
Compared to CNN-only and CNN–LSTM wearable health monitoring,” IEEE Sensors
baselines, the CNN–TCN-based Tiny Temporal Journal, vol. XX, no. X, pp. XX–XX, 20XX.
Transformer demonstrates: [2] B. Author et al., “Efficient transformers for
time-series analysis,” IEEE Transactions on
Improved early disease detection Biomedical Engineering, vol. XX, no. X, pp. XX–
sensitivity XX, 20XX.
Lower false alarm rates [3] C. Author et al., “Embedded AI and TinyML
for healthcare wearables,” IEEE Access, vol.
Comparable or lower latency under XX, pp. XX–XX, 20XX.
embedded constraints
VI. Discussion
The integration of efficient self-attention
enhances the model’s ability to detect subtle
physiological changes that precede overt
disease symptoms. Unlike cloud-dependent
systems, the proposed architecture enables
fully on-device inference, improving privacy,
reliability, and responsiveness.
The hybrid CNN–TCN–Transformer design
strikes a balance between expressive power
and computational efficiency, making it
particularly suitable for long-term wearable
deployment.
VII. Conclusion
This report presented a CNN–TCN-based Tiny
Temporal Transformer with efficient self-
attention for embedded AI-based smart