Embedded Artificial Intelligence
by
Dr. Dubacharla Gyaneshwar
Indian Institute of Information Technology Raichur (IIIT-R)
AD320 - F-slot - 2026
Course Contents
•Foundations of Embedded AI
•Computational and Architectural
Foundations
•Model Optimization
•Vision and Audio AI
•Applications and Case studies
Dr. Dubacharla Gyaneshwar
Course Evaluation
•Evaluation Criteria:
❑80% Attendance or more: 5% (1-6)
❑2 Quizzes: 20% (1-5) - completed
❑Assignments: 10% (3-6)
❑Project: 25% (1-6)
❑End-semester: 40% (3-6 end)
•Dates of the exams or any changes will be
notified in advance through an email.
Dr. Dubacharla Gyaneshwar
Glossary / Keywords / Index Terms
•Embedded Systems •Profiling
•Artificial Intelligence •Training
•Computing •Inference
•Latency •Simulation
•Energy Efficiency •Accuracy
•Constraints •Memory
Dr. Dubacharla Gyaneshwar
Embedded Artificial Intelligence
• Embedded AI seamlessly integrates intelligence into everyday devices.
• It brings AI closer to the data source, enabling faster and context-
aware decision making.
• This enables enhanced efficiency and unlocks new possibilities across
diverse domains.
• With global data expected to exceed 181 zettabytes by 2025,
embedded AI helps overcome cloud latency and inefficiency.
Dr. Dubacharla Gyaneshwar
Embedded Systems
• An embedded system is a specialized computer system designed to
perform dedicated functions within larger devices.
• Hardware and software tightly integrated, often operating in real-time
or near-real-time
• Optimized for low power consumption, limited memory, and
constrained resources
• Widely used in automotive, healthcare, consumer electronics,
industrial machines, and IoT applications
Dr. Dubacharla Gyaneshwar
?
•Identify five everyday
devices that use embedded
systems.
•List their specifications
Dr. Dubacharla Gyaneshwar
Motivation
• Embedded Systems: fixed, rule-based control.
• Embedded AI: adaptive, learning-based intelligence.
• They provide automation, reliability, and efficiency across diverse domains.
• Embedded AI extends these systems with adaptive, learning-based intelligence.
• By processing data closer to the source, embedded AI reduces latency and
enhances responsiveness.
• It enables scalability, resilience, and smarter decision-making in a data-heavy
world.
• Together, embedded systems and embedded AI shift devices from simple
automation to true autonomy.
Dr. Dubacharla Gyaneshwar
?
•Suggest how embedded AI could enhance
each device’s functionality.
•Compare the benefits and limitations of
embedded AI vs traditional embedded
systems in your examples.
•Reflect on real-world implications:
efficiency, scalability, and user experience.
Dr. Dubacharla Gyaneshwar
?
•Given a list of devices: Car ECU, Smartwatch,
Microwave, Smart Speaker, Drone.
•Add two more devices of your choice
•Classify each as:
• Embedded System only
• Embedded AI enabled
• Both
•Why?
Dr. Dubacharla Gyaneshwar
Foundations of EAI
• An embedded AI (EAI) system can be modeled as a layered
architecture combining hardware, software, and AI algorithms.
• Typically represented as a pipeline of data acquisition,
preprocessing, AI inference, actuation/control, all optimized for
resource-constrained environments.
• An EAI system can be modeled as:
Dr. Dubacharla Gyaneshwar
Example timing model
Local versus cloud AI: timing model
• Embedded AI:
• Cloud AI:
• Cloud AI fails real-time constraints if,
Dr. Dubacharla Gyaneshwar
?
•Identify five EAI systems and prepare a table
with the following columns:
Signal, Data, Decision.
•List their specifications accordingly
Dr. Dubacharla Gyaneshwar
Key Embedded Constraints
• Resource: Limited CPU power, restricted memory, and strict energy
efficiency.
• Real-Time: Deterministic responses with RTOS-based scheduling to
meet deadlines.
• Cost: Affordable hardware and scalable, cost-effective production.
• Reliability & Safety: Fault tolerance and long life-cycle for mission-
critical use.
• Environmental: Operates under extreme conditions with compact,
lightweight form factors.
Dr. Dubacharla Gyaneshwar
Example Embedded Devices
• Examples include:
• Microcontrollers (MCUs) – e.g., ARM Cortex M, Arduino, ESP32
• Digital Signal Processors (DSPs) – optimized for audio, image, and
real time signal processing
• System-on-Chip (SoC) – integrates CPU, GPU, memory, and AI
accelerators (e.g., Qualcomm Snapdragon, Raspberry Pi SoC)
• Sensors – temperature, pressure, accelerometer, camera modules
• Actuators – motors, relays, servos for physical control
• Edge AI Accelerators – NPUs, TPUs, FPGAs designed for on device
inference
Dr. Dubacharla Gyaneshwar
Programming Options
• Low Level Languages:
• C / C++ – Most common for microcontrollers, efficient and close to
hardware.
• Assembly – Used for performance critical or ultra low level control.
• High Level / Scripting Languages
• Python– Easy prototyping, supported on boards like ESP32, Raspberry Pi.
• Java / Kotlin – Sometimes used in embedded Android systems.
• Domain Specific Frameworks
• Arduino IDE – Simplified C/C++ for Arduino boards.
• MATLAB/Simulink – Model based design, auto code generation for
DSP/MCU.
• RTOS & Embedded OS Support
• FreeRTOS, Zephyr, Embedded Linux – Provide multitasking, scheduling, and
device drivers.
• AI/ML Deployment Options
• TensorFlow Lite, ONNX Runtime, Edge Impulse – Optimized libraries for
running ML models on microcontrollers and NPUs.
Dr. Dubacharla Gyaneshwar
Syntax Recap
Write the syntax for the following
•C program
•Python Program
Dr. Dubacharla Gyaneshwar
Syntax Recap
Dr. Dubacharla Gyaneshwar
Embedded Program
• An embedded program is typically structured differently
from a general-purpose application.
• What makes a program “Embedded”?
• An embedded program typically has:
• Initialization – sets up hardware, peripherals, and variables.
• Infinite Loop – continuously runs to monitor and control the system.
• Sensor → Decision → Actuation – processes input signals, makes
decisions, and drives actuators.
• Minimal Human Interface – no reliance on keyboard, mouse, or
display; often interacts through sensors and simple outputs.
Dr. Dubacharla Gyaneshwar
?
Write a simple program that demonstrate:
• C program for a rule-based temperature
monitoring system that continuously reads a
simulated sensor value and turns a fan ON or
OFF based on a fixed threshold.
Dr. Dubacharla Gyaneshwar
?
Write a simple program that demonstrate:
• Modify the same C program by replacing the
rule-based logic with a simple AI decision
function (ex: a linear equation) and take the
control decision based on its output.
Dr. Dubacharla Gyaneshwar
?
Write a simple program that demonstrate:
• Write a Python program that simulates the same
embedded AI behavior using functions for
sensing and decision-making, running locally
without user input or internet access.
Dr. Dubacharla Gyaneshwar
Memory Constraint in EAI
• What is the memory constraint?
• Embedded devices have limited: RAM and Flash
• In EAI, models must fit entirely inside the device
• Memory is used for:
• Model parameters (weights)
• Intermediate activations
• Input data buffers
• Example:
• an AI model: 1 million parameters
• 32-bit storage:1,000,000×4 = 4 MB
Dr. Dubacharla Gyaneshwar
?
A 5-layer fully connected (dense) neural network
has the following structure:
• Input layer: 128 neurons
• Hidden Layer 1: 64 neurons
• Hidden Layer 2: 64 neurons
• Hidden Layer 3: 32 neurons
• Output layer: 10 neurons
Check whether the model can fit into an embedded device with 512 KB
Flash memory.
Dr. Dubacharla Gyaneshwar
?
Estimate the total parameters layer-wise, memory
layer-wise (32-bit float) and total memory size.
Write the condition for real-time constraint
Dr. Dubacharla Gyaneshwar
Computation Constraint in EAI
• What is the computation constraint?
• Embedded devices have limited: Clock speed and Parallelism
• AI computation = MAC operations
• Example:
• Simple neural layer:
• 256 inputs -> 256 MACs
• CNN layers may require millions of MACs
• If a device can do: 10 million MAC/sec
• Model requires: 50 million MACs
• Take away is “A model may fit in memory but still be too slow to run.”
Dr. Dubacharla Gyaneshwar
?
A 5-layer fully connected (dense) neural network has the following
structure: Input layer: 128 neurons
Hidden Layer 1: 64 neurons
Hidden Layer 2: 64 neurons
Hidden Layer 3: 32 neurons
Output layer: 10 neurons
One MAC operation per weight per inference
• Calculate the number of MAC operations required for one inference.
• If the embedded processor supports 5 million MACs per second, calculate the maximum
inference rate.
• Comment on whether the system can support real-time inference at 50 inferences per
second.
Dr. Dubacharla Gyaneshwar
Latency Constraint in EAI
• What is latency?
• Time between sensing and decision
• Embedded systems often have deadlines
•Example:
• Obstacle-avoidance robot
• Allowed reaction time = 50 ms
• AI inference time = 80 ms
• Cloud comparison:
• Cloud adds upload + download delay. Latency become uncertain.
• Take away is “Missing a deadline = system failure, not “slow output””
Dr. Dubacharla Gyaneshwar
Power & Energy Constraint in EAI
• What is power constraint?
• Embedded devices are: battery-powered & energy-limited
• AI-inference consumes energy every time it runs.
• Example:
• Smartwatch AI runs:
• 2000 times/day
• High-power model:
• Battery drains in hours
• Trade-off:
• High accuracy -> high energy
• Lower accuracy -> longer battery life
• Take away is “EAI optimizes accuracy per watt, not accuracy alone”.
Dr. Dubacharla Gyaneshwar
Thank You
Dr. Dubacharla Gyaneshwar