REAL TIME EMBEDDED SYSTEMS- ECE355 & PROMPT

ENGINEERING- CS321

Project Title: AutoSort Bin

BY

Fatma Ouahada

Habib Al Kateb

Rayen Chaibi

Mohammed Malik Darraji

Omar Ben Tkhayat

ACADEMIC SUPERVISORS

Professor Noura Baccar

Professor Abdjalil Laabed

LAB INSTRUCTORS

Professor Chedi Hamrouni

Professor Nourhene Akrouti

2025-2026
 ABSTRACT

This project presents the design and implementation of a Smart Waste Sorting Trash
Bin based on an STM32 microcontroller platform. The system seamlessly integrates
mechanical actuation, sensing, visual feedback, and artificial intelligence to
automate the waste disposal and sorting process, reducing human intervention and
improving recycling accuracy. A servo motor manages the opening and closing of
the bin lid, while a stepper motor rotates internal compartments to align with the
detected waste type. The fill level of each bin is continuously monitored using an
HC-SR04 ultrasonic sensor, with red and green LEDs providing immediate visual
feedback to indicate whether the bins are full or available.

The system incorporates a camera-based AI classification module that captures

images of waste items and accurately categorizes them into plastic, metal, or other
types. The classification results are transmitted to the STM32 microcontroller via
Ethernet in JSON format, enabling precise control of the stepper motor to direct
each waste item into the correct compartment. This integration of AI-driven decision
making with real-time embedded control demonstrates the potential of intelligent,
automated systems in modern waste management.

Overall, the project highlights a modular, scalable, and intelligent approach to

smart urban waste management, combining embedded hardware, firmware
development, and AI technologies. It provides a practical solution for improving
recycling efficiency, enhancing hygiene, and supporting sustainable practices in
smart city environments.

ii
 Table of Contents
ABSTRACT .................................................................................................................. ii
1. INTRODUCTION .................................................................................................. 1
2. SYSTEM DESIGN.................................................................................................. 2
2.1 Components of the system ....................................................................... 2
2.2 Design ......................................................................................................... 3
2.3 System Description .................................................................................... 4
2.3.1 circuit design ............................................................................................... 4
2.3.2 Simulation ................................................................................................. 5
2.3.3 Real Implementation ............................................................................... 5
2.4 Artificial Intelligence Classification System ................................................. 6
2.4.1 Purpose and Role of the AI System ........................................................ 6
2.4.2 Input Data Acquisition ............................................................................. 7
2.4.3 AI Model Architecture ............................................................................. 7
2.4.4 Classification Output and Decision Logic ............................................. 8
2.4.5 Communication with Embedded System .............................................. 8
3. SYSTEM DIAGRAMS AND REQUIREMENTS ............................................................ 9
3.1 System Requirements ..................................................................................... 9
3.1.1 Functional and Non-Functional Requirements ..................................... 9
3.1.2 Requirement Satisfaction Explanation ................................................... 9
3.2 SysML Diagrams ............................................................................................ 10
3.2.1 Structure Diagrams ................................................................................ 10
3.2.2 Behaviour Diagrams ......................................................................... 16
4. CODE IMPLEMENTATION .................................................................................... 20
4.1 Setup .............................................................................................................. 20
4.1.1 STM32CubeMX Configuration ............................................................... 20
4.1.2 Library Integration .................................................................................. 22
4.1.3 Mathematical Equations ....................................................................... 22
4.2 Code Explanation......................................................................................... 23
4.2.1 Embedded Firmware Implementation................................................. 23

iii
 4.2.2 AI Software Implementation ................................................................. 28
5. TESTING ................................................................................................................ 32
5.1 Servo Motor Testing (Lid Mechanism) ........................................................ 32
5.2 Ultrasonic Sensor Testing (Fill Level)............................................................ 33
5.3 Camera Module and AI Integration Testing .............................................. 33
5.4 Stepper Motor Testing (Sorting Mechanism) .............................................. 35
6. RESULTS ................................................................................................................ 36
7. SUMMARY............................................................................................................ 37
7.1 Research Goals and Achievements........................................................... 37
7.2 Challenges and Solutions ............................................................................ 37
7.3 Future Enhancements .................................................................................. 38
8. REFERENCES......................................................................................................... 39

iv
 1. INTRODUCTION
Effective waste management is a critical component of sustainable urban
development, as improper disposal and inefficient sorting contribute significantly to
environmental pollution and resource loss. Traditional trash bins rely heavily on
manual sorting, which is not only time-consuming but also prone to human error,
often leading to contamination of recyclable materials and increased waste in
landfills. Moreover, conventional bins provide limited feedback to users and offer no
support for hygienic handling, further complicating effective waste management in
densely populated areas.

This project addresses these challenges by designing an automated smart trash bin
system that leverages STM32-based embedded systems and Artificial Intelligence to
streamline waste segregation and minimize human interaction. By integrating an
OV7670 camera module with an external AI classification module, the system
identifies waste types in real time and directs them to specific compartments via
precise stepper and servo motor control. The software architecture, developed
using STM32CubeMX and HAL libraries, emphasizes modularity and scalability for
future smart city integration.

Ultimately, this intelligent solution enhances urban sanitation by improving recycling

accuracy, ensuring hygienic waste management through sensor-driven automation,
and promoting sustainable environmental practices.

1
 2. SYSTEM DESIGN
2.1 Components of the system

Figure 1: STM32 Microcontroller Figure 2: Red + green LEDS Figure 3: OV7670 Camera Module

Figure 4: 17HS4401 stepper motor Figure 5: S3003 Servo motor

Figure 6:HC-SR04 sensor Figure 7: driver motor A4988

2
Component Description Role

STM32F4 Microcontroller Main controller Central processing and

control
Servo Motor (S3003) PWM-controlled motor Lid opening/closing

Stepper Motor + Driver Precision actuator Waste compartment rotation

HC-SR04 Ultrasonic Sensor Distance sensor Fill-level detection

Red LED Visual indicator Bin full

Green LED Visual indicator Bin not full

Camera Module Image acquisition Waste image capture

AI Model Classification system Waste type detection

Ethernet Interface Communication JSON data transfer

2.2Design

Figure 8: 3D system design Figure 9: 3D system design

Figure 10: 3D system design

3
 2.3 System Description
2.3.1 circuit design
Pin usage configured in STM32CubeMX:

Pin Mode Function

PA8 TIM1_CH1 (PWM) Servo motor signal

PA0 GPIO Output Servo test/control

PA1 GPIO Output Servo test/control

PA2 GPIO Output HC-SR04 TRIG

PA3 GPIO Input HC-SR04 ECHO

PA4 GPIO Input Stepper completion signal

PB0 GPIO Output Red LED

PB1 GPIO Output Green LED

PD13 GPIO OUTPUT Driver motor STEO pin

PD12 GPIO OUTPUT Driver motor DIR pin

PC6 Alternate Function DCMI_D0

PC7 Alternate Function DCMI_D1

PC8 Alternate Function DCMI_D2

PC9 Alternate Function DCMI_D3

PE4 Alternate Function DCMI_D4

PB6 Alternate Function DCMI_D5

PE5 Alternate Function DCMI_D6

PE6 Alternate Function DCMI_D7

4
Timers, GPIO modes, and clock prescalers were configured using STM32CubeMX,
while application logic was written using HAL functions

2.3.2 Simulation

Figure 11: Circuit Design

2.3.3 Real Implementation

Figure 12: real implementation of the circuit

5
 Figure 13: Camera Module implementation

2.4 Artificial Intelligence Classification System

The Artificial Intelligence (AI) system is responsible for automatically identifying the
type of object/material captured by the camera. Its main role is to replace manual
inspection by performing a fast and consistent classification from an input image.
The embedded system is mainly used for image acquisition and transmission, while
the heavy computation (model inference) is executed on a local AI server to
achieve higher performance and easier model updates. The AI system outputs a
predicted class (e.g., metal, plastic, organic) along with a confidence score that
indicates how certain the model is.

2.4.1 Purpose and Role of the AI System

The input data for the AI system is an image captured using an OV7670 camera
connected to the STM32F439ZI microcontroller through the DCMI interface and
DMA. The camera provides frames in QVGA resolution (320×240) and the image can
be transferred in RGB565 format (2 bytes per pixel) or converted to another format if
required.

Once a capture condition is met (trigger from a sensor or event logic), the
embedded system captures a single frame and stores it in a memory buffer. The
frame is then transmitted over Ethernet using TCP to a local AI server. The server
reconstructs the image from the received byte stream and performs preprocessing
(resizing/normalizing) before inference.

6
2.4.2 Input Data Acquisition

The input data for the AI system is an image captured using an OV7670 camera
connected to the STM32F439ZI microcontroller through the DCMI interface and
DMA. The camera provides frames in QVGA resolution (320×240) and the image can
be transferred in RGB565 format (2 bytes per pixel) or converted to another format if
required.

Once a capture condition is met (trigger from a sensor or event logic), the
embedded system captures a single frame and stores it in a memory buffer. The
frame is then transmitted over Ethernet using TCP to a local AI server. The server
reconstructs the image from the received byte stream and performs preprocessing
(resizing/normalizing) before inference.

2.4.3 AI Model Architecture

The AI classification model is designed to work with visual features extracted from
the captured images. A lightweight Convolutional Neural Network (CNN) is typically
used because CNNs are highly effective for image classification tasks. The model
may be either:

• A custom CNN trained specifically on the project dataset, or

• A transfer-learning model (pretrained backbone such as MobileNet-like
architecture) fine-tuned on the target classes.

The model generally consists of:

1. Feature extraction layers (convolutions + pooling) to learn edges, textures,

shapes, and patterns.
2. Intermediate layers to combine low-level features into higher-level object
representations.
3. Classification head (fully connected / global average pooling + dense layer)
producing probabilities for each class.

The architecture is optimized to balance accuracy and speed, since inference must
be fast enough for real-time or near real-time decision making.

7
2.4.4 Classification Output and Decision Logic

After the AI server receives the image frame and completes preprocessing, the
model performs inference and produces a single predicted class label (for example:
plastic, metal, or organic).

In this project, the decision logic is intentionally simple:

• The top predicted class label is selected as the final output.

• This label is treated as the system’s decision and is sent back to the
embedded system.

Since we do not use confidence scores in the current implementation, the

embedded system relies only on the returned label to continue the next action (such
as sorting, logging, or triggering another mechanical step).

2.4.5 Communication with Embedded System

Communication between the embedded system and the AI server is performed

using Ethernet (TCP) for reliability and ordered delivery of image data. The
embedded system sends:

1. A small header (image width, height, format, payload length)

2. The raw image payload (RGB565 bytes)

The AI server responds with a compact message containing:

• the predicted class ID or name 12 This division of tasks makes the

embedded firmware simpler and ensures the AI model can be
updated on the server side without modifying the embedded
hardware.

8
3. SYSTEM DIAGRAMS AND REQUIREMENTS
3.1 System Requirements

3.1.1 Functional and Non-Functional Requirements

ID Requirement Type Description

FR1 Functional Automatic lid opening

FR2 Functional Waste classification

FR3 Functional Mechanical sorting

FR4 Functional Fill-level detection

FR5 Functional Visual status indication

NFR1 Non-Functional Real-time response

NFR2 Non-Functional Reliability

NFR3 Non-Functional Safety

3.1.2 Requirement Satisfaction Explanation

• FR1 achieved using PWM-controlled servo motor.

• FR2 achieved using AI model and Ethernet communication.

• FR3 achieved using stepper motor positioning.

• FR4 achieved using HC-SR04 distance measurement.

• FR5 achieved using LED indicators.

9
3.2 SysML Diagrams
SysML (Systems Modeling Language) is a graphical language for modeling complex
systems, ideal for representing the architecture, behavior, and interactions within the
alarm system that we are developing. By using SysML, we can model the system's
components, how they interact, and the system's requirements. This approach will
provide a clear and structured way to design our embedded alarm system, ensuring
that the system operates as intended and meets its requirements.

3.2.1 Structure Diagrams

[Link] Block Definition Diagram: Shows the main components of the Smart
Waste Sorting System and how they relate conceptually

Figure 14: Block Definition Diagram

10
 Block summary

Block Name Type Description / Responsibility

Top-level system that integrates

sensing, actuation, AI
SmartWasteSortingSystem System classification, and
communication to automate
waste sorting.
Central control unit. Reads
Hardware
sensors, receives classification
STM32Microcontroller /
data, controls motors and LEDs,
Controller
and manages system timing.

Controls the opening and

ServoMotor Actuator closing of the trash bin lid using
PWM signals.

Rotates the internal sorting

mechanism to align the correct
StepperMotor Actuator
waste compartment based on
classification results.

Measures the distance between

UltrasonicSensor (HC-
Sensor the sensor and trash level to
SR04)
detect whether the bin is full.

Output / Provides continuous visual status

LEDIndicators Visual of the bin using red and green
Feedback LEDs.

Captures an image of the waste

CameraModule Sensor
object placed inside the bin.

Processes the captured image

AIClassificationSystem Software / AI and classifies waste as plastic,
metal, or other.

Transfers classification results in

EthernetCommunication Communication JSON format from the AI system
to the STM32 microcontroller.

11
 Relationship summary

Source Block Target Block Relationship Type Description

The system is composed

SmartWaste-
STM32Microcontroller Composition of an STM32 that acts as
SortingSystem
the central controller.

The servo motor is a

SmartWaste- physical part of the
ServoMotor Composition
SortingSystem system used for lid
control.

The stepper motor is

SmartWaste- part of the system and
StepperMotor Composition
SortingSystem performs waste sorting
rotation.

The ultrasonic sensor is

SmartWaste-
UltrasonicSensor Composition integrated to monitor
SortingSystem
trash level.

LED indicators are

SmartWaste-
LED Indicators Composition included for visual
SortingSystem
system feedback.

Camera module is part

SmartWaste--
CameraModule Composition of the system for image
SortingSystem
acquisition.

AI system is included to
SmartWaste- AIClassificationSyste
Composition enable intelligent waste
SortingSystem m
classification.

12
 Ethernet
SmartWaste- Ethernet communication is a
Composition
SortingSystem Communication subsystem of the overall
system.

STM32 sends PWM

STM32Micro-
ServoMotor Dependency signals to control lid
controller
movement.

STM32 controls stepper

STM32Micro-
StepperMotor Dependency rotation using GPIO
controller
signals.

STM32 triggers and

STM32Micro- reads distance
UltrasonicSensor Dependency
controller measurements via
TRIG/ECHO pins.

STM32Micro- STM32 drives LEDs using

LEDIndicators Dependency
controller GPIO outputs.

AI system sends
AIClassificatio
Ethernet classification results
n Dependency
Communication using JSON over
System
Ethernet.

Ethernet STM32 receives

Communicatio STM32Microcontroller Dependency classification results for
n decision-making.

13
[Link] Internal block diagram : Shows how the components interact and connect
internally, illustrating the flow of signals and control.

Figure 15: Internal block diagram

• Components and their roles:

STM32 Microcontroller

• Acts as the central processing and control unit.

• Receives sensor data, stepper completion signals, and AI classification results.

• Controls the servo motor, stepper motor, and LED indicators.

14
Servo Motor (Lid Control)

• Opens and closes the trash bin lid.

• Activated only after the stepper motor completes its rotation.

• Controlled using PWM generated by the STM32.

Stepper Motor (Sorting Mechanism)

• Rotates internal compartments to align with the classified waste type.

• Sends a digital completion signal to the STM32 after finishing rotation.

Ultrasonic Sensor (HC-SR04)

• Measures the distance between the sensor and trash level.

• Enables detection of a full bin using a predefined distance threshold.

Red and Green LEDs

• Provide real-time visual feedback.

• Red LED indicates the bin is full.

• Green LED indicates the bin is not full.

Camera Module

• Captures images of the waste object after insertion.

AI Classification Module

• Processes captured images using a trained machine learning model.

• Classifies waste into plastic, metal, or other categories.

Ethernet Communication Module

• Transmits classification results in JSON format to the STM32.

• Enables real-time communication between AI system and embedded

controller.

15
3.2.1 Behaviour Diagrams

[Link] Use Case Diagram:

Shows the interactions between the system and external actors (User, Sensors,
AI module), and the main functions the system performs, such as detecting
trash type, rotating the stepper motor, opening the lid, and updating bin
status.

Figure 16: Use Case Diagram

Actors

• User:

o Inserts waste into the bin.

o Observes bin status via LED indicators.

• System (Smart Waste Sorting System):

o Performs automatic sorting of waste.

o Processes images for AI classification.

Use Cases

• Insert Waste: User deposits waste into the bin.

• View Waste Status: User observes whether the bin is full or empty via LEDs.

16
 • Automatic Sorting: System rotates internal compartments to align with the
appropriate waste type.

• Monitor AI Classification: System captures images and classifies waste type

using AI.

Include Relationships

• Insert Waste includes Automatic Sorting:

o Every time the user deposits waste, the system automatically sorts it
based on AI classification results.

[Link] Sequence Diagram:

Illustrates the chronological flow of operations in the system, from the moment
the user places trash into the system, through AI analysis, motor control, and
bin status update via sensors and LEDs. This diagram highlights the interaction
between all system components.

Figure 17: Sequence Diagram

Actors / Components
• User

o Places trash into the system.

o Observes bin status via LEDs (red = full, green = not full).
• Camera
o Captures an image of the trash.
o Sends the image to the AI module for analysis.

17
 • AI Module
o Processes the camera image.
o Determines the trash type (e.g., plastic, organic, paper).
o Sends the trash type information to the system.
• System / Controller
o Receives trash type from the AI module.
o Commands stepper and servo motors to handle the trash.
o Monitors motor completion signals.
• Stepper Motor (17HS4401)
o Rotates the mechanism to align the correct bin for the trash.
• Servo Motor
o Opens and closes the top lid of the selected bin so the trash can fall in.
• Bin Sensor
o Detects whether each bin is full or not.
o Sends status to LEDs.
• LEDs
o Show bin status (red = full, green = not full).
o Controlled only by sensors, not by the system.
Process Flow

1. User places trash into the system.

2. Camera captures an image of the trash.

3. AI module analyzes the image and determines the trash type.

4. AI sends trash type data to the system.

5. System commands stepper motor to rotate to the corresponding bin.

6. Stepper motor rotates, system waits for rotation completion.

7. System signals servo motor to open the bin lid.

8. User drops trash into the selected bin.

9. Servo motor closes lid, system receives completion signal.

10. Bin sensors detect fill level of bins.

11. LEDs update: red if bin is full, green if not full.

12. System remains idle until next trash placement.

18
[Link] Activity Diagram:

Represents the detailed workflow of the system’s processes, including decision

points such as detecting bin fullness, selecting the correct bin based on AI
output, and completing the trash deposit operation. It provides a visual map
of system behaviour and process flow.

Figure 18: Activity Diagram

19
4. CODE IMPLEMENTATION
This chapter describes the software implementation of the smart waste sorting
system. It covers the configuration of the microcontroller, the libraries used, the
mathematical models applied, and a detailed explanation of the main program
logic.

4.1 Setup

4.1.1 STM32CubeMX Configuration

The STM32F439 microcontroller was configured using STM32CubeMX to generate the
project structure and initialize the hardware peripherals. The following configurations
were applied:

• Clock Configuration:

o Internal High-Speed Oscillator (HSI) was selected as the clock source.

o Phase-Locked Loop (PLL) was enabled to achieve a higher system

clock frequency.

o The system clock (SYSCLK) was configured to ensure stable timing for
PWM generation and sensor measurements.

Figure 19: Clock configuration in CubeMX

20
 • GPIO Configuration:

o GPIO pins were configured as output push-pull for:

▪ Red and green status LEDs.

▪ Ultrasonic sensor trigger pin.

▪ Stepper motor status output.

▪ driver motor STEO & DIR pins

o GPIO pins were configured as input for:

▪ Ultrasonic sensor echo pin.

Figure 20: Pins configuration in CubeMX

• Timer Configuration (TIM1):

o Timer 1 was configured in PWM mode.

o The timer generates a 50 Hz PWM signal required for servo motor

control.

o The PWM signal is used to control the opening and closing of the bin
lid.

All configurations were generated manually by STM32CubeMX and later completed

inside STM32CubeIDE.

21
4.1.2 Library Integration
The project uses the STM32 HAL (Hardware Abstraction Layer) libraries, which simplify
peripheral configuration and improve code portability. The following libraries are
integrated:

• main.h: Contains system-level definitions and peripheral handles.

• stm32f4xx_hal_gpio.h: Used for GPIO configuration and control.

• stm32f4xx_hal_tim.h: Used for timer and PWM control.

• stdint.h: Provides fixed-width integer types for reliable embedded

programming.

Additionally, the DWT (Data Watchpoint and Trace) cycle counter is enabled to allow
precise microsecond-level timing required for ultrasonic distance measurement.

4.1.3 Mathematical Equations

Several mathematical relationships are used in the system:

1. Servo Motor PWM Pulse Width Calculation

The servo angle is controlled by varying the PWM pulse width:

Angle × 1000
Pulse Width = 1000 + ( )
180

Where:

• Pulse width is in microseconds

• Angle is between 0° and 180°

2. Ultrasonic Distance Calculation

The distance is calculated based on the time-of-flight of the ultrasonic wave:
𝑡×𝑣
Distance (cm) =
2

Where:

• 𝑡is the echo pulse duration in microseconds

• 𝑣 = 0.0343 cm/µsis the speed of sound

• Division by 2 accounts for the round-trip travel of the sound wave

22
4.2 Code Explanation

4.2.1 Embedded Firmware Implementation

The software is structured into clear functional blocks to improve readability

and maintainability.

• Initialization Phase:

o The HAL library and system clock are initialized.

o GPIO pins and Timer 1 are configured.

o PWM output is started for servo motor control.

o The servo motor is initially set to the closed position.

o The DWT cycle counter is enabled for precise timing.

Figure 21: Initialization Phase

• Servo Control Logic:

o A dedicated function converts the desired angle into a PWM

duty cycle.

Figure 22: Angle Conversion Function

23
 o The servo opens the lid when the stepper motor signals that the
correct bin position is ready.

o After a short delay, the lid closes automatically.

Figure 23: Lid Control

• Stepper Motor Logic:

Converts a desired rotation angle into an equivalent number of motor steps.

o Enables angular control without using position sensors.

o Ensures accurate alignment of the sorting mechanism.

Figure 24: Step calculation

Sets the motor rotation direction by driving the DIR pin of the A4988 driver.

o Generates a defined number of STEP pulses corresponding to the desired

angular displacement.
o Each STEP pulse causes the motor to advance by one mechanical step
(1.8°).
o The delay between pulses controls the motor speed and prevents step loss or
vibration.
o The function enables open-loop position control by counting pulses instead
of using feedback sensors

24
 Figure 25 Stepper Control

o Resets the PA4 signal to indicate that a new sorting operation has
started.
o Commands the stepper motor to rotate to the angle associated with
the detected waste material.
o Sets PA4 high once the rotation is completed, signaling successful bin
alignment.
o Introduces a delay to allow the waste item to fall into the correct bin.
o Returns the motor to its initial position to prepare the system for the
next sorting cycle.

Figure 26: Sorting Sequence

25
• Ultrasonic Sensor Processing:

o The trigger pin sends a short pulse to initiate distance

measurement.

o The echo pulse duration is measured using the DWT cycle

counter.

o The measured distance represents the fill level of the bin.

Figure 27: Fill Level Detection

• LED Status Indication:

o If the measured distance is below the defined threshold, the bin

is considered full and the red LED is turned on.

o If the distance is above the threshold, the bin is not full and the
green LED is turned on.

26
 Figure 28: Bin Status Display

• Key Functions

Servo_SetAngle(uint8_t angle)
– Converts a desired angle to PWM duty cycle and moves the servo.
read_ultrasonic_distance(void)
– Triggers the ultrasonic sensor, measures echo time, and returns
distance in cm.
SystemClock_Config(void)
– Configures the system clock.
MX_TIM1_Init(void)
– Initializes Timer 1 for PWM output.
MX_GPIO_Init(void)
– Configures GPIO pins for LEDs, stepper simulation, and ultrasonic
sensor.
Error_Handler(void)
– Infinite loop in case of errors
angle_to_steps(uint16_t angle)
– Converts a rotation angle into the corresponding number of motor
steps.
step_motor(uint16_t steps, GPIO_PinState direction)
– Rotates the stepper motor by a specified number of steps in a given
direction using step pulses.
• Main Loop Execution:

o The system continuously monitors the stepper signal and

ultrasonic sensor.

o All actions are performed in real time with a short delay to

ensure stable operation.

27
4.2.2 AI Software Implementation

[Link] Artificial Intelligence Classification System

Figure 29: Ollama API Integration and Structured JSON Prompt Construction

generate_with_ollama Function : sends text and image data to a local Ollama API to
generate AI responses. It manages the asynchronous connection, processes
streaming chunks of data, and handles the conversion of the final result into a
structured JSON format.

Response Processing: It uses an asynchronous HTTP client to stream the response

from the server, stitching together text chunks and error-checking the status codes in
real-time

28
[Link] Purpose and Role of the AI System

API endpoint role: receives frame bytes → sends to AI model → returns classification):
(from [Link] — the endpoint that acts as the AI “server” side classifier

Figure 30:Integrated API Endpoint for Automated Image-to-Material Classification

29
[Link] Input Data Acquisition

A) Receive raw bytes + verify it’s a valid image

Figure 31:Image Validation and Pre-processing Service

B) Convert received bytes to base64 for Ollama

Figure 32: Base64 Image Encoding

C) Compute basic visual features (used to support prompt reasoning)

Figure 33:Visual Data Analysis and Feature Extraction

30
[Link] Classification Output and Decision Logic
This module serves as the "instructional brain" of the AI pipeline, ensuring the raw
image data is translated into a precise sorting command. It operates through the
following key mechanisms:

• Persona & Context Enforcement: It defines the AI as an "expert materials

analyst" and injects computed image features (like brightness and texture)
directly into the prompt to ground the model’s decision in physical data.

• Strict Whitelist Validation: The code defines a closed set of "allowed" material
categories—such as metal, plastic, and organic—preventing the AI from
returning ambiguous or unsupported classifications.

• JSON Output Contract: It enforces a rigid data structure requiring exactly

three keys: material, confidence, and reason. This eliminates conversational
"filler" and ensures the output is immediately processable by the system's
decision-making logic.

• Decision Transparency: By requiring a "reason" field, the code forces the AI to

justify its classification based on visual cues, which is essential for debugging
and improving accuracy during testing.

Figure 34:Automated Decision Logic and Material Classification

31
5. TESTING
This section details the functional verification of the individual hardware components
and the software integration. Each component was tested against its design
requirements to ensure reliability before full system assembly

5.1 Servo Motor Testing (Lid Mechanism)

• Objective: To verify that the PWM signal correctly controls the bin lid's rotation
and that the logic prevents premature opening.

• Procedure: A PWM signal with varying duty cycles was sent to the servo to test
the 0° (closed) and 90° (open) positions.

• Validation :

Figure 35: Lid in initial position (0°) Figure 36: Lid started opening Figure 37: Lid opened (90°)

Figure 38: Lid rotating back to 0° Figure 39: Lid closed (0°)

• Observations: The lid successfully opened and closed based on the

microcontroller commands.

32
5.2 Ultrasonic Sensor Testing (Fill Level)

• Objective: To validate the accuracy of distance measurement and the

responsiveness of the status LEDs.

• Procedure: Objects were placed at varying distances from the sensor to

simulate different waste levels.

• Validation :

Figure 40: Sensor detected bin not full Figure 41: Sensor detected full bin

• Observations: The system correctly identified "Full" status when an object was
within the threshold, triggering the Red LED. It returned to "Available" (Green
LED) when the object was far .

5.3 Camera Module and AI Integration Testing

• Objective: To evaluate the image capture quality (OV7670) and the
classification accuracy of the AI model.

• Procedure: A physical object (a metal can) was placed in front of the

camera . The camera captured a 320 X 240 frame, which was sent to the AI
model using the prompt building and API logic.

• Validation:

Figure 42: Photo taken with OV7670 Camera

33
 Figure 43: Image transfer

Figure 44: Classification using the AI model

• Observations: Partial Success / Failure. While the image transfer and API
communication were successful, the AI misclassified the metal can as "Plastic"
in the most recent trial, despite succeeding in all previous trials.

• Root Cause Analysis: This discrepancy likely stems from lighting conditions or
the low-resolution QVGA capture, which can confuse the AI's feature
extraction. Further refinement of the build_material_prompt logic or the
"Reason" key in the JSON response is required to debug this.

34
5.4 Stepper Motor Testing (Sorting Mechanism)
• Objective: To verify the step accuracy and torque required for the sorting
movement.

• Procedure: Commands were sent to the stepper driver to rotate the sorting
tray to specific coordinates.

• Observation: Inconsistent Actuation. The motor successfully initiates

movement and follows the programmed "pattern" (vibrating in sync with the
code pulses), but it frequently fails to complete a full or half rotation. The
motion is characterized by significant vibration and "skipping," where the
internal magnetic rotor fails to lock onto the next step.
• Technical Assessment: This behaviour suggests a power-to-timing mismatch.
The vibration indicates that the motor is receiving the signals (the "pattern"),
but either the current limit on the driver is set too low to create enough
holding torque, or the pulse frequency is too high for the motor to keep up,
causing it to "stall" mid-sequence.

Figure 45: Stepper Motor Drive System Assembly

35
6. RESULTS
The system demonstrated high reliability in its core control logic. The servo motor
responded correctly to microcontroller triggers for lid operation, status LEDs reflected
the bin’s fill level accurately, and ultrasonic distance measurements were consistent
with physical measurements.

However, the integrated components required further calibration. The camera

module successfully captured and transmitted images to the AI system, but
classification results showed inconsistency, such as misidentifying metallic objects as
plastic due to lighting glare and QVGA resolution limits. Similarly, while the stepper
motor correctly followed the programmed pulse patterns, it experienced significant
vibration and mid-sequence stalling. This was attributed to insufficient driving current
or pulse-timing mismatches, preventing the motor from achieving the torque
necessary for smooth, full rotations.

Figure 46: AutoSort's Final System Prototype and Hardware Integration

36
7. SUMMARY
7.1 Research Goals and Achievements
The project successfully integrated embedded systems, sensing, and AI-driven
decision-making by bridging hardware control with high-level data analysis. Using
the STM32, the system captures environmental data and converts it into structured
information that provides the necessary context for classification. This data is
processed through specialized prompt engineering, ensuring the AI produces a
machine-readable response instead of unpredictable text. This protocol allows the
local model to act as an expert analyst, delivering precise material identifications
that the system can immediately process. Ultimately, the research achieved a full
closed-loop automation where AI decisions are converted into physical sorting
movements via a stepper motor drive

7.2 Challenges and Solutions

The main challenges encountered during the design and implementation of the
automatic sorting bin, along with their corresponding solutions, are summarized as
follows:
• Stepper motor vibration and unstable rotation: The issue was caused by
insufficient motor current and was resolved by properly adjusting the A4988
current limit and using a stable 12 V power supply.

• Incorrect current measurement during debugging: Using the milliampere

range on the multimeter limited the current; switching to the high-current (10
A) range allowed correct motor operation.

• Unpredictable driver behavior due to floating control pins: Floating ENABLE

and micro stepping pins were stabilized by connecting them to defined logic
levels (ENABLE to GND, MS pins fixed).

• Voltage instability and electrical noise affecting the motor driver: This was
mitigated by adding two decoupling capacitors between VCC and GND,
ensuring voltage stability and protecting the driver from current spikes.

• Synchronization between motor motion and system logic: A completion

signal on PA4 was added to indicate the end of motor rotation and
coordinate with other system components.

37
7.3 Future Enhancements
Advanced Motor Control: Implement closed-loop control using encoders on the
stepper motor to achieve precise positioning and reduce vibration without relying
solely on open-loop timing and current adjustments.

Power Management Optimization: Integrate a more robust power management

system with regulated voltage rails and battery monitoring to ensure stable
operation under varying load conditions.

Improved Sensor Fusion: Combine multiple sensors (ultrasonic, IR, or weight sensors)
to increase accuracy in fill-level detection and object presence, reducing false
triggers.

Enhanced AI Integration: Upgrade the AI model to perform on-board classification

using lightweight neural networks, reducing latency and dependency on an
external server.

Wireless Communication: Add Wi-Fi or Bluetooth connectivity to allow remote

monitoring, notifications for full bins, and integration into a larger smart city waste
management system.

User Feedback and Interface: Introduce an LCD or touch interface to provide real-
time status, system logs, or manual override controls for maintenance and
troubleshooting.

Energy Efficiency Improvements: Explore low-power modes for the microcontroller

and sensors when the system is idle to save energy in battery powered
implementation

38
8. REFERENCES
[1] I. Sosunova and J. Porras, "IoT-Enabled Smart Waste Management Systems for
Smart Cities: A Systematic Review," IEEE Access, vol. 10, pp. 73326–73363, 2022. doi:
10.1109/ACCESS.2022.3188308.

[2] “SysML Diagram Tutorial | [Link],” [Link]. [Link]

diagram tutorial/

[3] “Internal Block Diagram - an overview | ScienceDirect Topics,”

[Link]. [Link]
science/internal-block-diagram

[4] Lucidchart, “UML Use Case Diagram Tutorial,” [Link], 2019.

[Link]

[5] “Activity Diagram - Activity Diagram Symbols, Examples, and More,”

[Link], 2022. [Link]
diagram/?srsltid=AfmBOopnx1JYfkDJyf3CRIk79Bt1jLEZfFrN5pu0UXcX4JBrpHBkp0zo

[6] STMicroelectronics, “STM32F4 Series Reference Manual”, RM0090, 2016. [Online].

Available: [Link]
[Link]

[7] Seeed Studio, “HC-SR04 Ultrasonic Sensor Datasheet”, 2015. [Online]. Available:
[Link]

[8] Futuba, “Standard Servo S3003 Datasheet”. [Online]. Available:

[Link]

[9] Ollama, "Ollama: Local Large Language Models," 2024. [Online]. Available:
[Link]

39