Embedded Artificial Intelligence
by
Dr. Dubacharla Gyaneshwar
Indian Institute of Information Technology Raichur (IIIT-R)
AD320 - F-slot - 2026
Course Contents
•Foundations of Embedded AI
•Computational and Architectural
Foundations
•Model Optimization
•Vision and Audio AI
•Applications and Case studies
Dr. Dubacharla Gyaneshwar
Course Evaluation
•Evaluation Criteria:
❑80% Attendance or more: 5% (1-6)
❑2 Quizzes: 20% (1-5) - completed
❑Assignments: 10% (3-6)
❑Project: 25% (1-6)
❑End-semester: 40% (3-6 end)
•Dates of the exams or any changes will be
notified in advance through an email.
Dr. Dubacharla Gyaneshwar
Project work
•Evaluation Criteria:
❑Project: 25% (1-6)
• Project Objectives
• Intermediate updates of work
• Demonstration
Dr. Dubacharla Gyaneshwar
Glossary / Keywords / Index Terms
•Embedded Systems •Profiling
•Artificial Intelligence •Training
•Computing •Inference
•Latency •Simulation
•Energy Efficiency •Accuracy
•Constraints •Memory
Dr. Dubacharla Gyaneshwar
Embedded Artificial Intelligence
• Embedded AI seamlessly integrates intelligence into everyday devices.
• It brings AI closer to the data source, enabling faster and context-
aware decision making.
• This enables enhanced efficiency and unlocks new possibilities across
diverse domains.
• With global data expected to exceed 181 zettabytes by 2025,
embedded AI helps overcome cloud latency and inefficiency.
Dr. Dubacharla Gyaneshwar
Embedded Systems
• An embedded system is a specialized computer system designed to
perform dedicated functions within larger devices.
• Hardware and software tightly integrated, often operating in real-time
or near-real-time
• Optimized for low power consumption, limited memory, and
constrained resources
• Widely used in automotive, healthcare, consumer electronics,
industrial machines, and IoT applications
Dr. Dubacharla Gyaneshwar
Motivation
• Embedded Systems: fixed, rule-based control.
• Embedded AI (EAI): adaptive, learning-based intelligence.
• They provide automation, reliability, and efficiency across diverse domains.
• EAI extends these systems with adaptive, learning-based intelligence.
• By processing data closer to the source, EAI reduces latency and enhances
responsiveness.
• It enables scalability, resilience, and smarter decision-making in a data-heavy
world.
• Together, embedded systems and EAI shift devices from simple automation to
true autonomy.
Dr. Dubacharla Gyaneshwar
Embedded Intelligence
• Sense–Decide–Act: The Core of Embedded Intelligence.
• EAI systems operate in a continuous real-time loop rather than executing one-
time predictions. E.g., smart fan, gesture-controlled light, voice wake word, etc.
• The system first senses the physical environment using sensors such as
temperature sensors, cameras, microphones, or motion detectors.
• The sensed data is then processed by an EAI model which interprets the input
and makes a decision.
• Based on this decision, the system performs an action using actuators such as
motors, LEDs, fans, alarms, or control signals.
Dr. Dubacharla Gyaneshwar
Example
• Smart Fan Control System:
• In a traditional embedded system, the temperature sensor
value is compared against a fixed threshold using if–else
conditions to turn the fan ON or OFF.
• In an EAI system, multiple sensor inputs such as
temperature, humidity, and room occupancy are fed into a
trained AI model.
• The model learns optimal fan control behavior from
historical data and produces a smooth and adaptive control
output.
• This allows better comfort, energy efficiency, and
robustness to noisy sensor readings compared to rigid
threshold-based logic.
Dr. Dubacharla Gyaneshwar
Example
• Embedded System Example (Traditional):
• Setup: A microcontroller (like Arduino or STM32) connected to a
temperature sensor (e.g., LM35, DHT11) and a fan motor driver.
• Logic:
• If temperature < 25°C → Fan OFF
• If 25–30°C → Fan at low speed
• If 30–35°C → Fan at medium speed
• If > 35°C → Fan at high speed
• Characteristics:
• Rule-based, deterministic control.
• No learning or adaptation.
• Simple threshold comparisons coded directly into firmware.
• This is a classic embedded system: fixed logic, predictable behavior, no
intelligence beyond programmed rules.
Dr. Dubacharla Gyaneshwar
Example
• EAI Example:
• Setup: Same hardware, but with an AI/ML model deployed on the
microcontroller or edge device.
• Logic:
• The system collects historical temperature and fan speed data.
• A lightweight ML model (e.g., regression, decision tree, or TinyML neural
network) predicts optimal fan speed based on patterns.
• It adapts to environment changes (e.g., learns that in a crowded room, fan
needs to ramp up earlier even if temperature is only 28°C).
• Characteristics:
• Predictive and adaptive control.
• Can optimize for energy efficiency, comfort, or noise reduction.
• Learns from data instead of relying only on fixed thresholds
• This is embedded AI: intelligence at the edge, enabling smarter,
context-aware decisions.
Dr. Dubacharla Gyaneshwar
?
•When is a simple embedded system
sufficient?
•What advantages could EAI bring to a fan
speed control system?
•How do we judge if adding AI is “overdoing”
it?
•What risks or trade-offs come with using
embedded AI in IoT devices?
Dr. Dubacharla Gyaneshwar
EAI
• EAI is not required for simple, predictable environments (like
a single room with stable temperature). It becomes valuable
when:
• The environment is complex or variable,
• Energy optimization is critical,
• Personalization is needed, or User Preference Learning
• Predictive capabilities (comfort or maintenance) add real
benefit
• Example: AI detects unusual fan speed vs. temperature patterns
that may indicate motor wear or sensor malfunction.
• This allows preventive maintenance before failure occurs.
Dr. Dubacharla Gyaneshwar
Foundations of EAI
• AI as a Learned Decision Function:
• In Embedded AI, the decision-making block is represented
as a mathematical function.
• where 𝑥 represents sensor inputs such as temperature,
images, or audio features, and 𝑦 represents the control
output or class label.
• The function 𝑓 is not manually written by the programmer
but is learned automatically from training data using
machine learning algorithms.
• Once trained, the function is deployed onto the embedded
device and executed locally for real-time inference.
• This enables the system to generalize to new situations
that were not explicitly programmed.
Dr. Dubacharla Gyaneshwar
Foundations of EAI
• An embedded AI (EAI) system can be modeled as a layered
architecture combining hardware, software, and AI algorithms.
• Typically represented as a pipeline of data acquisition,
preprocessing, AI inference, actuation/control, all optimized for
resource-constrained environments.
• An EAI system can be modeled as:
Dr. Dubacharla Gyaneshwar
Example
Local versus cloud AI: timing model
• Embedded AI:
• Cloud AI:
• Cloud AI fails real-time constraints if,
Dr. Dubacharla Gyaneshwar
Example Devices
• Examples include:
• Microcontrollers (MCUs) – e.g., ARM Cortex M, Arduino, ESP32
• Digital Signal Processors (DSPs) – optimized for audio, image, and
real time signal processing
• System-on-Chip (SoC) – integrates CPU, GPU, memory, and AI
accelerators (e.g., Qualcomm Snapdragon, Raspberry Pi SoC)
• Sensors – temperature, pressure, accelerometer, camera modules
• Actuators – motors, relays, servos for physical control
• Edge AI Accelerators – NPUs, TPUs, FPGAs designed for on device
inference
Dr. Dubacharla Gyaneshwar
Memory Constraint
• What is the memory constraint?
• Embedded devices have limited: RAM and Flash
• In EAI, models must fit entirely inside the device
• Memory is used for:
• Model parameters (weights)
• Intermediate activations
• Input data buffers
• Example:
• an AI model: 1 million parameters
• 32-bit storage:1,000,000×4 = 4 MB
Dr. Dubacharla Gyaneshwar
Computation Constraint
• What is the computation constraint?
• Embedded devices have limited: Clock speed and Parallelism
• AI computation = MAC operations
• It can also be expressed in floating-point operations (FLOPs)
• Example:
• Simple neural layer:
• 256 inputs -> 256 MACs
• CNN layers may require millions of MACs
• If a device can do: 10 million MAC/sec
• Model requires: 50 million MACs
• Take away is “A model may fit in memory but still be too slow to run.”
Dr. Dubacharla Gyaneshwar
MAC Operations
• What is a MAC operation?
• In digital signal processing and EAI contexts, MAC operations
refer to Multiply–Accumulate operations.
• They are the fundamental building blocks of many
algorithms, especially in filtering, convolution, and neural
networks
• 1 MAC • 6 MACs = 2 times of 3 mul and 2 add
Dr. Dubacharla Gyaneshwar
?
• A neural network has 4 inputs and 1 output neuron?
• An input image of size 3×3 is convolved with 1 filter of size
3×3, stride 1, no padding
• An input image of size 7×7×3 is convolved with 6 filters of
size 3×3, stride 1, no padding.
• A 1D signal of length 10 with 2 channels is convolved using 4
filters of size 3, stride 1, no padding.
Dr. Dubacharla Gyaneshwar
Programming Options
• Low Level Languages:
• C / C++ – Most common for microcontrollers, efficient and close to
hardware.
• Assembly – Used for performance critical or ultra low level control.
• High Level / Scripting Languages
• Python– Easy prototyping, supported on boards like ESP32, Raspberry Pi.
• Java / Kotlin – Sometimes used in embedded Android systems.
• Domain Specific Frameworks
• Arduino IDE – Simplified C/C++ for Arduino boards.
• MATLAB/Simulink – Model based design, auto code generation
for DSP/MCU.
• RTOS & Embedded OS Support
• FreeRTOS, Zephyr, Embedded Linux – Provide multitasking, scheduling,
and
device drivers.
• AI/ML Deployment Options
• TensorFlow Lite, ONNX Runtime, Edge Impulse – Optimized libraries
for running ML models on microcontrollers and NPUs.
Dr. Dubacharla Gyaneshwar
Syntax Recap
Dr. Dubacharla Gyaneshwar
?
• Write C programs for the following:
• K-NN
• K-means
• Perceptron
• Whether the above 3 techniques qualify for EAI usage
Dr. Dubacharla Gyaneshwar
Computational Foundations
• MAC Operations (Multiply–Accumulate):
• Core unit of AI computation; how convolution, matrix
multiplication, and dot products boil down to MACs.
• Complexity Analysis:
• How input size, kernel size, channels, and layers scale MAC
counts.
• Numerical Precision:
• Floating point vs. fixed point; why embedded AI often uses
INT8 or even binary/ternary weights.
• Dataflow Models:
• How data moves through layers (convolution, pooling, fully
connected).
• Energy–Accuracy Trade offs:
• How reducing precision or pruning reduces MACs but may
affect accuracy.
Dr. Dubacharla Gyaneshwar
Architectural Foundations
• Microcontrollers & DSPs:
• ARM Cortex M, TI DSPs — optimized for low power signal
processing and AI inference.
• Edge AI Accelerators:
• TPUs, NPUs, Qualcomm Hexagon DSP — specialized blocks for
parallel MAC execution.
• Memory Hierarchy:
• SRAM, cache, flash — importance of minimizing data
movement (often more costly than computation).
• Parallelism & SIMD:
• Single Instruction Multiple Data, pipelining, vector instructions
for faster inference.
• Frameworks & Libraries:
• TensorFlow Lite Micro, CMSIS NN — software stacks that map
AI models onto constrained hardware.
Dr. Dubacharla Gyaneshwar
Discussions
•List of Machine Learning and Deep
Learning models
Dr. Dubacharla Gyaneshwar
Tools
• Raspberry Pi comes with preloaded
languages like Python, Scratch and C or C++
pre-installed.
Dr. Dubacharla Gyaneshwar
Thank You
Dr. Dubacharla Gyaneshwar