Embedded Artificial Intelligence
by
Dr. Dubacharla Gyaneshwar
Indian Institute of Information Technology Raichur (IIIT-R)
AD320 - F-slot - 2026
Course Contents
•Foundations of Embedded AI
•Computational and Architectural
Foundations
•Model Optimization
•Vision and Audio AI
•Applications and Case studies
Dr. Dubacharla Gyaneshwar
Course Evaluation
•Evaluation Criteria:
❑80% Attendance or more: 5% (1-6)
❑2 Quizzes: 20% (1-5) - completed
❑Assignments: 10% (3-6)
❑Project: 25% (1-6)
❑End-semester: 40% (3-6 end)
•Dates of the exams or any changes will be
notified in advance through an email.
Dr. Dubacharla Gyaneshwar
Project work
•Evaluation Criteria:
❑Project: 25% (1-6)
• Project Objectives
• Intermediate updates of work
• Demonstration
Dr. Dubacharla Gyaneshwar
Glossary / Keywords / Index Terms
•Embedded Systems •Profiling
•Artificial Intelligence •Training
•Computing •Inference
•Latency •Simulation
•Energy Efficiency •Accuracy
•Constraints •Memory
Dr. Dubacharla Gyaneshwar
Embedded Artificial Intelligence (EAI)
• EAI seamlessly integrates intelligence into everyday devices.
• It brings AI closer to the data source, enabling faster and context-
aware decision making.
• This enables enhanced efficiency and unlocks new possibilities across
diverse domains.
• With global data expected to exceed 181 zettabytes by 2025,
embedded AI helps overcome cloud latency and inefficiency.
Dr. Dubacharla Gyaneshwar
Embedded Intelligence
• Sense–Decide–Act: The Core of Embedded Intelligence.
• EAI systems operate in a continuous real-time loop rather than executing one-
time predictions. E.g., smart fan, gesture-controlled light, voice wake word, etc.
• The system first senses the physical environment using sensors such as
temperature sensors, cameras, microphones, or motion detectors.
• The sensed data is then processed by an EAI model which interprets the input
and makes a decision.
• Based on this decision, the system performs an action using actuators such as
motors, LEDs, fans, alarms, or control signals.
Dr. Dubacharla Gyaneshwar
Example
• Smart Fan Control System:
• In a traditional embedded system, the temperature sensor
value is compared against a fixed threshold using if–else
conditions to turn the fan ON or OFF.
• In an EAI system, multiple sensor inputs such as
temperature, humidity, and room occupancy are fed into a
trained AI model.
• The model learns optimal fan control behavior from
historical data and produces a smooth and adaptive control
output.
• This allows better comfort, energy efficiency, and
robustness to noisy sensor readings compared to rigid
threshold-based logic.
Dr. Dubacharla Gyaneshwar
Example
• Embedded System Example (Traditional):
• Setup: A microcontroller (like Arduino or STM32) connected to a
temperature sensor (e.g., LM35, DHT11) and a fan motor driver.
• Logic:
• If temperature < 25°C → Fan OFF
• If 25–30°C → Fan at low speed
• If 30–35°C → Fan at medium speed
• If > 35°C → Fan at high speed
• Characteristics:
• Rule-based, deterministic control.
• No learning or adaptation.
• Simple threshold comparisons coded directly into firmware.
• This is a classic embedded system: fixed logic, predictable behavior, no
intelligence beyond programmed rules.
Dr. Dubacharla Gyaneshwar
Example
• EAI Example:
• Setup: Same hardware, but with an AI/ML model deployed on the
microcontroller or edge device.
• Logic:
• The system collects historical temperature and fan speed data.
• A lightweight ML model (e.g., regression, decision tree, or TinyML neural
network) predicts optimal fan speed based on patterns.
• It adapts to environment changes (e.g., learns that in a crowded room, fan
needs to ramp up earlier even if temperature is only 28°C).
• Characteristics:
• Predictive and adaptive control.
• Can optimize for energy efficiency, comfort, or noise reduction.
• Learns from data instead of relying only on fixed thresholds
• This is embedded AI: intelligence at the edge, enabling smarter,
context-aware decisions.
Dr. Dubacharla Gyaneshwar
EAI
• EAI is not required for simple, predictable environments (like
a single room with stable temperature). It becomes valuable
when:
• The environment is complex or variable,
• Energy optimization is critical,
• Personalization is needed, or User Preference Learning
• Predictive capabilities (comfort or maintenance) add real
benefit
• Example: AI detects unusual fan speed vs. temperature patterns
that may indicate motor wear or sensor malfunction.
• This allows preventive maintenance before failure occurs.
Dr. Dubacharla Gyaneshwar
Foundations of EAI
• AI as a Learned Decision Function:
• In Embedded AI, the decision-making block is represented
as a mathematical function.
• where 𝑥 represents sensor inputs such as temperature,
images, or audio features, and 𝑦 represents the control
output or class label.
• The function 𝑓 is not manually written by the programmer
but is learned automatically from training data using
machine learning algorithms.
• Once trained, the function is deployed onto the embedded
device and executed locally for real-time inference.
• This enables the system to generalize to new situations
that were not explicitly programmed.
Dr. Dubacharla Gyaneshwar
Foundations of EAI
• An embedded AI (EAI) system can be modeled as a layered
architecture combining hardware, software, and AI algorithms.
• Typically represented as a pipeline of data acquisition,
preprocessing, AI inference, actuation/control, all optimized for
resource-constrained environments.
• An EAI system can be modeled as:
Dr. Dubacharla Gyaneshwar
Example Devices
• Examples include:
• Microcontrollers (MCUs) – e.g., ARM Cortex M, Arduino, ESP32
• Digital Signal Processors (DSPs) – optimized for audio, image, and
real time signal processing
• System-on-Chip (SoC) – integrates CPU, GPU, memory, and AI
accelerators (e.g., Qualcomm Snapdragon, Raspberry Pi SoC)
• Sensors – temperature, pressure, accelerometer, camera modules
• Actuators – motors, relays, servos for physical control
• Edge AI Accelerators – NPUs, TPUs, FPGAs designed for on device
inference
Dr. Dubacharla Gyaneshwar
Key Embedded Constraint
• Resource: Limited CPU power, restricted memory, and strict energy
efficiency.
• Real-Time: Deterministic responses with RTOS-based scheduling to
meet deadlines.
• Cost: Affordable hardware and scalable, cost-effective production.
• Reliability & Safety: Fault tolerance and long life-cycle for mission-
critical use.
• Environmental: Operates under extreme conditions with compact,
lightweight form factors.
Dr. Dubacharla Gyaneshwar
Memory Constraint
• What is the memory constraint?
• Embedded devices have limited: RAM and Flash
• In EAI, models must fit entirely inside the device
• Memory is used for:
• Model parameters (weights)
• Intermediate activations
• Input data buffers
• Example:
• an AI model: 1 million parameters
• 32-bit storage:1,000,000×4 = 4 MB
Dr. Dubacharla Gyaneshwar
Computation Constraint
• What is the computation constraint?
• Embedded devices have limited: Clock speed and Parallelism
• AI computation = MAC operations
• It can also be expressed in floating-point operations (FLOPs)
• Example:
• Simple neural layer:
• 256 inputs -> 256 MACs
• CNN layers may require millions of MACs
• If a device can do: 10 million MAC/sec
• Model requires: 50 million MACs
• Take away is “A model may fit in memory but still be too slow to run.”
Dr. Dubacharla Gyaneshwar
MAC Operations
• What is a MAC operation?
• In digital signal processing and EAI contexts, MAC operations
refer to Multiply–Accumulate operations.
• They are the fundamental building blocks of many
algorithms, especially in filtering, convolution, and neural
networks
• 1 MAC • 6 MACs = 2 times of 3 mul and 2 add
Dr. Dubacharla Gyaneshwar
Latency Constraint
• What is latency?
• Time between sensing and decision
• Embedded systems often have deadlines
•Example:
• Obstacle-avoidance robot
• Allowed reaction time = 50 ms
• AI inference time = 80 ms
• Cloud comparison:
• Cloud adds upload + download delay. Latency become uncertain.
• Take away is “Missing a deadline = system failure, not “slow output””
Dr. Dubacharla Gyaneshwar
Power & Energy Constraints
• What is power constraint?
• Embedded devices are: battery-powered & energy-limited
• AI-inference consumes energy every time it runs.
• Example:
• Smartwatch AI runs:
• 2000 times/day
• High-power model:
• Battery drains in hours
• Trade-off:
• High accuracy -> high energy
• Lower accuracy -> longer battery life
• Take away is “EAI optimizes accuracy per watt, not accuracy alone”.
Dr. Dubacharla Gyaneshwar
Computational Foundations
• MAC Operations (Multiply–Accumulate):
• Core unit of AI computation; how convolution, matrix
multiplication, and dot products boil down to MACs.
• Complexity Analysis:
• How input size, kernel size, channels, and layers scale MAC
counts.
• Numerical Precision:
• Floating point vs. fixed point; why embedded AI often uses
INT8 or even binary/ternary weights.
• Dataflow Models:
• How data moves through layers (convolution, pooling, fully
connected).
• Energy–Accuracy Trade offs:
• How reducing precision or pruning reduces MACs but may
affect accuracy.
Dr. Dubacharla Gyaneshwar
Architectural Foundations
• Microcontrollers & DSPs:
• ARM Cortex M, TI DSPs — optimized for low power signal
processing and AI inference.
• Edge AI Accelerators:
• TPUs, NPUs, Qualcomm Hexagon DSP — specialized blocks for
parallel MAC execution.
• Memory Hierarchy:
• SRAM, cache, flash — importance of minimizing data
movement (often more costly than computation).
• Parallelism & SIMD:
• Single Instruction Multiple Data, pipelining, vector instructions
for faster inference.
• Frameworks & Libraries:
• TensorFlow Lite Micro, CMSIS NN — software stacks that map
AI models onto constrained hardware.
Dr. Dubacharla Gyaneshwar
Support Vector Machine
Dr. Dubacharla Gyaneshwar
Perceptron
Dr. Dubacharla Gyaneshwar
Activation Function
Dr. Dubacharla Gyaneshwar
Types of Propagation
Dr. Dubacharla Gyaneshwar
Perceptron learning
Dr. Dubacharla Gyaneshwar
Deep Convolution Neural Networks
Dr. Dubacharla Gyaneshwar
?
• Represent a small dataset (features + labels) and identify
whether it’s suitable for classification or regression.
• Distinguish between supervised and unsupervised
learning using examples.
• State the difference between training and inference
phases.
• Compute basic operations (distance, mean, dot product,
convolution) that form the building blocks of these
algorithms.
Dr. Dubacharla Gyaneshwar
?
• Sketch a separating line for a simple 2D dataset.
• Perform one convolution operation on a small image
patch with a given kernel.
• In KNN, show how storing the dataset is the “training”
phase.
• In K-means, perform one iteration of centroid
initialization and update.
Dr. Dubacharla Gyaneshwar
Inference architectures
• Key considerations for EAI:
• Embedded constraints: Limited memory, compute, and power
budgets (often <1W, RAM in MBs, storage in GBs).
• Resource estimation: Multiply–accumulate (MAC) counts,
parameter size, latency targets.
• Hardware effects: CPU vs GPU vs DSP vs NPU availability,
quantization support, cache sizes, and bandwidth.
• Which AI models are feasible under these limits?
• How Each Fits Embedded AI (Inference View Only)
Dr. Dubacharla Gyaneshwar
Inference architectures
K-NN (Heavy Memory, Simple Compute)
• Inference behavior:
• Store all training samples
• Compute distance to every point
• Embedded insight:
• Very high memory
• High latency as data grows
• Simple math
• Great example of what does NOT scale on embedded
systems
Dr. Dubacharla Gyaneshwar
Inference architectures
K-NN (Heavy Memory, Simple Compute)
Dr. Dubacharla Gyaneshwar
Inference architectures
K-means (used for clustering, inference = assignment)
Dr. Dubacharla Gyaneshwar
Inference architectures
Perceptron
Dr. Dubacharla Gyaneshwar
Inference architectures
Support Vector Machine (SVM)
Dr. Dubacharla Gyaneshwar
Inference architectures
Support Vector Machine (SVM)
Dr. Dubacharla Gyaneshwar
Inference architectures
Convolution Neural Network (CNN)
Dr. Dubacharla Gyaneshwar
?
• Compute the memory usage and MAC operations for both the above
questions
Dr. Dubacharla Gyaneshwar
?
• Compute the memory usage and MAC operations for both the above
questions
Dr. Dubacharla Gyaneshwar
Model Optimization
• Typical model optimization in CNNs and ML reduces loss and improves
accuracy by tuning parameters and architecture.
• Techniques include
• Optimization algorithms (optimizers): algorithms like SGD, Adam, etc.
• Hyperparameter tuning: Adjusting settings before training, such as learning rate,
batch size
• Model architecture search: For CNNs, choosing the right structure (e.g., depth, filter
sizes, etc.) and techniques like ResNet or transfer learning to improve performance.
• Regularization & optimization:
• Dropout Layers
• Batch Normalization
• Early stopping
• Many more….
Dr. Dubacharla Gyaneshwar
Model Optimization
• But in Embedded AI, the priorities shift…
• Embedded AI Optimization Goals:
• Memory footprint → Fit models into limited RAM/Flash
• Latency → Real-time inference on-device
• Computation → Efficient use of limited CPU/DSP resources
• Power/Energy → Extend battery life, reduce heat
• Techniques:
• Model compression (quantization, pruning, knowledge distillation)
• Hardware-aware optimization (edge accelerators, DSPs, NPUs)
• Efficient architectures (MobileNet, TinyML models)
• Trade-off analysis: accuracy vs. efficiency
Dr. Dubacharla Gyaneshwar
Model Optimization
• What is Model Compression?
• In short: Traditional optimization = accuracy focus. Embedded optimization = efficiency
focus. Model compression is the bridge
Dr. Dubacharla Gyaneshwar
Model Compression
• Key techniques: For instance in the case of DNN
Dr. Dubacharla Gyaneshwar
Foundations
•?
Dr. Dubacharla Gyaneshwar
?
Dr. Dubacharla Gyaneshwar
Foundations
Dr. Dubacharla Gyaneshwar
Foundations
Dr. Dubacharla Gyaneshwar
Foundations
Dr. Dubacharla Gyaneshwar
?
Dr. Dubacharla Gyaneshwar
Pruning
Dr. Dubacharla Gyaneshwar
Pruning
Dr. Dubacharla Gyaneshwar
Quantization
Dr. Dubacharla Gyaneshwar
Algorithms
Dr. Dubacharla Gyaneshwar
?
Dr. Dubacharla Gyaneshwar
?
Dr. Dubacharla Gyaneshwar
?
Dr. Dubacharla Gyaneshwar
Thank You
Dr. Dubacharla Gyaneshwar