Gesture-Controlled Robot for Medical

Assistance Using Laptop Camera, ESP32,

and Motor Driver
submitted in partially fulfilment of the requirements for the degree of

Bachelor of Technology
in

Electronics and Communication Engineering

by
Isha Srivastava (2100970310072)
Ishika Raj (2100970310073)
Harshvardhan (2100970310065)
Under the Guidance of
Ms. Nahid Malik
Assistant Professor
Department of Electronics and Communication Engineering ([Link] ECE –
Accredited by NBA), Galgotias College of Engineering and Technology,
Greater Noida. (Affiliated to Dr.A.P.J Abdul Kalam Technical University,
Lucknow)

May, 2025

i
 DECLARATION
We hereby declare that the thesis entitled “Gesture-Controlled Robot for Medical
Assistance Using Laptop Camera, ESP32, and Motor Driver” submitted by us, for the
award of the degree of Bachelor of Technology in Electronics and Communication
Engineering to Galgotias College of Engineering and Technology, Greater Noida
affiliated to Dr. A.P.J. Abdul Kalam Technical University, Lucknow is a record of
bonafide work carried out by us under the supervision of Ms. Nahid Malik.
We further declare that the work reported in this thesis has not been submitted and
will not be submitted, either in part or in full, for the award of any other degree or
diploma in this institute or any other institute or university.

Place : Greater Noida

Date : Signature of the Candidates

ii
 CERTIFICATE

This is to certify that the thesis entitled “Gesture-Controlled Robot for Medical
Assistance Using Laptop Camera, ESP32, and Motor Driver” submitted by Ishika
Raj, Isha Srivastava and Harshvardhan , Department of Electronics and Communication
Engineering, Galgotias College of Engineering and Technology, Greater Noida affiliated
to Dr. A.P.J. Abdul Kalam Technical University (AKTU), Lucknow, for the award of the
degree of Bachelor of Technology in Electronics and Communication Engineering, is a
record of bonafide work carried out by them under my supervision, as per AKTU code of
academic and research ethics.

The contents of this report have not been submitted and will not be submitted
either in part or in full, for the award of any other degree or diploma in this institute or
any other institute or university. The thesis fulfils the requirements and regulations of the
University and in my opinion meets the necessary standards for submission.

Place: Greater Noida

Date: Signature of the Guide

The thesis is satisfactory / unsatisfactory

Internal Examiner 1 Internal Examiner 2

Approved by

Head of the Department

iii
 ABSTRACT
This project focuses on the development of a gesture-controlled robot specifically
designed to assist in medical environments where contactless operation is crucial,
such as isolation wards, quarantine zones, and pandemic-response units. The system
uses a vision-based human–machine interface, allowing users to control a mobile
robot through intuitive hand gestures captured by a laptop’s inbuilt camera. Using
Python with OpenCV and MediaPipe libraries, specific gestures—such as open palm,
closed fist, and directional pointing—are detected and mapped to movement
commands. These commands are sent via USB serial communication to an ESP32
microcontroller, which controls two DC motors through an L298N motor driver. This
enables the robot to execute movements like forward, backward, left, right, and stop
in real time.

The robot offers a contactless, hygienic, and cost-effective alternative to traditional

systems that rely on wearable sensors or manual input, making it especially valuable
for reducing infection risk during patient interaction. Built using low-cost
components, the system is portable, scalable, and easy to deploy in hospitals,
particularly in resource-constrained settings. Experimental testing shows accurate
gesture recognition and responsive control in well-lit environments. This solution
lays the groundwork for future medical robotics aimed at automating routine tasks
such as delivering medicines, documents, or equipment, thereby enhancing safety
and efficiency in clinical care.

This gesture-controlled robot is tailored to meet the growing demand for contactless
medical assistance in hospitals and quarantine zones. By enabling hands-free control
without wearable devices, it ensures a hygienic interface ideal for infection-prone
areas. The system is affordable, easy to deploy, and well-suited for automating
routine hospital tasks like medicine delivery or remote communication. Its reliable
performance in controlled environments highlights its potential for real-world
medical applications. This project provides a strong foundation for future innovations
in healthcare robotics.

iv
 ACKNOWLEDGEMENT
First of all, we would like to express our gratitude to our beloved Chairman Shri.
Suneel Galgotia and CEO Shri. Dhruv Galgotia for proving necessary facilities to
carry out and finish the project successfully. We are grateful to our Director Prof.
Vikram Bali for his support and encouragement.

Our special thanks goes to Dr. R L Yadav, Head of the Department, Electronics
and Communication Engineering, for providing an environment that encouraged us
in working towards our goal and supported in completion of our project. This project
gave us an opportunity to apply creative and critical thinking skills.

We would like to present our deep sense of gratitude to our project guide, Ms. Nahid
Malik Assistant Professor, Department of Electronics and Communication
Engineering who has always been a source of motivation and firm support for
carrying out the project. The supervision and support that he gave truly helped in the
progression of the project. His cooperation is much indeed appreciated. We are
highly obliged to him for his valuable advices and moral support during research
period.

Finally, our greatest and special gratitude goes to our family for their love and support.

Place: Greater Noida Ishika Raj

Isha Srivastava
Date: Harshvardhan

v
 TABLE OF CONTENTS
CHAPTER CONTENTS PAGE
NO. NO.

Title Page i

Declaration ii

Certificate iii

Abstract iv

Acknowledgement v

Table of Contents vi

List of Figures xi

List of Tables xii

List of Abbreviations xiii

1 Introduction 1

1.1 Project Overview 1

1.2 Background and Motivation 3

1.3 Objective 3

1.4 Problem Statement 5

1.5 Scope Of the Project 5

1.5.1 Key Features 6

[Link] Gesture Recognition Interface 6

[Link] Robotic Platform 6

[Link] Wireless Communication 6

[Link] Basic Navigation 6

[Link] Use Case Demonstration 6

vi
1.5.2 Future Scope 7

vii
 1.6 Summary 7

2 Literature review 9

2.1 Introduction 9

2.2 Gesture recognition Techniques 9

2.2.1 Sensor based Recognition 9

2.2.2 Vision based recognition 10

2.3 Robotic in Health Care 12

2.4 Gesture Controlled Robots 13

2.5 Comparative Study of Existing Systems 14

2.6 Research gaps and Motivation 20

2.7 Summary 20

3 System Analysis and Design 21

3.1 Introduction 21

3.2 Requirement Analysis 21

3.2.1 Functional Requirements 21

3.2.2 Non- Functional Requirements 21

3.3 System Architecture 22

3.3.1 Block Diagram 23

3.4 Detailed Module Description 23

3.4.1 Gesture Capture Module 23

3.4.2 Gesture Recognition Module 23

3.4.3 Command Transmission Module 23

3.4.4 Robot Control Module 23

3.4.5 Power Management 24

3.5 System Flow 24

viii
3.6 Hardware-Software Interaction 25

ix
 3.7 Challenge and Design Consideration 26

3.8 Summary 26

4 Implementation and Testing 22

4.1 Hardware Implementation 22

4.1.1 Laptop with In-Built Camera 22

4.1.2 ESP32 Microcontroller 27

4.1.3 L298N Motor Driver Module 31

4.1.4 DC Motors 31

4.1.5 Robot Chasis with Wheels 32

4.1.6 Jumper Wires 32

4.1.7 Power Source 32

4.2 Software Implementation 34

4.2.1 Gesture Recognition Using OpenCV 34

4.2.2 Communication with ESP32 38

4.2.3 ESP32 Control Code 39

4.3 Integration and Testing 39

4.3.1 Module Testing 39

4.3.2 End-to-End Testing 39

4.3.3 Test Results 39

4.4 Challenges and Solutions 39

4.5 Final Implementation Outcome 40

5 Result and discussion 41

5.1 Overview of Testing Process 41

5.2 Evaluation Parameters 41

5.3 Test Environment Setup 41

x
5.4 Performance Results 42

xi
 5.4.1 Gesture Recognition Accuracy 42

5.4.2 Command Latency 42

5.4.3 Robot Mobility and Control 43

5.4.4 Connectivity Stability 43

5.5 Limitations Observed 43

5.6 Discussion 43

6 Conclusion and future scope 46

6.1 Key Outcomes and Achievements 46

6.2 Limitations of the Project 47

6.3 Future Scope 48

6.3.1 Integration with AI and 48

Machine Learning

6.3.2 Incorporation of Voice Commands 48

and Multimodal Interaction

6.3.3 Autonomous Navigation and Obstacle 48

Avoidance

6.3.4 Expansion into Telemedicine and 48

Remote Operations

6.3.5 Use of Advanced Sensors and 49

Embedded Systems

6.3.6 Battery Optimization and Wireless 49

Charging

6.3.7 Enhanced Safety and Compliance 49

Features

6.3.8 Integration with Hospital Management 49

Systems (HMS)

6.3.9 Scalability for Mass Deployment 49

xii
 6.3.10 Applications Beyond Healthcare 50

6.4 Future Scope (Extended with 50

Advanced Technologies)

6.4.1 CAN Bus Integration for Modular 50

Communication

6.4.2 GPS for Outdoor Medical Logistics 50

6.4.3 Navigation LiDAR for 51

Autonomous Indoor Movement

6.4.4 Safety LiDAR for Human 51

Detection and Collision Avoidance

6.4.5 High-Resolution Camera Systems for 51

Enhanced Gesture and Scene
Recognition

6.4.6 Simultaneous Localization and 51

Mapping (SLAM)
6.4.7 Edge Computing for Real-Time 52
Decision Making
6.4.8 Mapping and Cloud Synchronization 52
6.4.9 Environmental Adaptability and Multi- 52
Zone Routing
6.4.10 Fail-Safe Systems and Redundancy 53
6.5 Summary 53

References

Appendix

xiii
 LIST OF FIGURES

Figure No. Title Page No.

1.1 System Architecture 4
3.1 Block Diagram of System Architecture 23
3.2 Flowchart of proposed method 25
3.3 Hardware-Software Interaction through ESP-32 25
4.1 Components for Hardware Implementation 27
4.2 ESP-32 microcontroller 27
4.3 Pin configuration of ESP-32 29
4.4 L298N Motor Driver Module 31
4.5 L293 D Motor 31
4.6 (a) Hardware Implementation (Top View) 32
4.6 (b) Hardware Implementation (Side View) 33
4.7 (a) (a) Forward Command Recognition 36
4.7 (b) (b) Stop Command Recognition 36
4.7 (c) (c) Backward Command Recognition 37
4.7 (d) (d) Left Command Recognition 38
4.7 (e) (e) Right Command Recognition 38

xiv
 LIST OF TABLES

Table No. Title Page No.

2.1 Literature study of Existing System 19
2.2 Comparison Study 19

3.1 Control Module 24

4.1 Comparison of ESP32 with other microcontrollers 29

4.2 Test result 39

5.1 Parameters for Testing 41

5.2 Gesture Accuracy 42

xv
 LIST OF ABBREVIATIONS

AI Artificial Intelligence
HMI Human-Machine Interaction
USB Universal Serial Bus
ESP32 Espressif System-on-Chip
DC Direct Current
PWM Pulse Width Module
GUI Graphical User Interface
IoT Internet of Things
Wi-Fi Wireless Fidelity
BLE Bluetooth Low Energy
UDP User Datagram Protocol
FPS Frames Per Second
HSV Hue, Saturation, Value (color model)
IDE Integrated Development Environment
ML Machine Learning
CNN Convolutional Neural Network
OpenCV Open Source Computer Vision Library
API Application Programming Interface
GPIO General Purpose Input Output
LAN Local area network
OBST Obstacle
ALU Arithmetic Logic unit
IC Integrated Circuit
L298N Dual H-Bridge Motor Driver IC
LED Light Emitting Diode
RGB Red Green Blue (color model)
PySerial Python Serial Library
DSO Digital Storage Oscilloscope

xvi
 CHAPTER 1
INTRODUCTION

The field of robotics is rapidly evolving, driven by advancements in artificial

intelligence, computer vision, and embedded systems. Among the various methods of
human-machine interaction, gesture recognition stands out due to its natural and
intuitive approach. Instead of relying on physical interfaces like keyboards or
joysticks, gesture-based control systems allow users to interact with machines using
simple hand movements, making them ideal for applications in robotics, smart
environments, and assistive technologies.

This project focuses on developing a gesture-controlled robot that can be operated

wirelessly through hand gestures captured by a laptop's inbuilt camera. Unlike
traditional systems that require additional hardware such as accelerometers or RF
modules, this system leverages computer vision techniques to interpret hand gestures
in real-time. The laptop processes video frames using OpenCV and MediaPipe
libraries to detect specific hand movements and translates them into control
commands.

In this context, the integration of computer vision with real-time robotics offers a
compelling opportunity to develop contactless, intelligent systems that can assist in
daily tasks across medical, industrial, and domestic environments. Vision-based
gesture control, in particular, is gaining popularity due to its non-reliance on physical
hardware like gloves, sensors, or wearables. According to Mallick & Kumar (2019)
and Chen et al. (2019), vision-based gesture recognition systems using OpenCV and
convolutional neural networks (CNNs) respectively have demonstrated high accuracy
and responsiveness, especially for applications where user convenience and minimal
hardware are critical.

1.1 Project Overview

This project proposes the development of a gesture-controlled robot for medical

assistance, designed to interpret real-time hand gestures using only a laptop’s inbuilt
camera. The system leverages OpenCV and MediaPipe, two widely adopted computer
vision libraries, to detect hand gestures with high efficiency and responsiveness. The
processed gesture commands are transmitted via socket-based Wi-Fi communication
to an ESP32 microcontroller, which interprets the commands and drives DC motors
1
using an L298N motor driver. This allows the robot to perform basic movements—
forward, backward, left, right, and stop—in real-time based on user gestures.

Unlike many traditional gesture-controlled robots that use accelerometer modules

(e.g., Yadav et al., IRJET 2017) or flex sensor gloves (Kumar et al., IJCAT 2023), our
system eliminates the need for additional wearable devices. This makes the solution
cost- effective, user-friendly, and more hygienic, aligning well with medical and
clinical applications where minimal contact is essential.

The key objectives of this project are:

 To design and implement a gesture-controlled robotic system for medical

assistance.
 To utilize computer vision techniques for real-time gesture recognition using
a laptop camera.
 To establish wireless communication between the gesture detection system
and the robot using Wi-Fi (socket programming).
 To enable basic movement controls (forward, backward, left, right, stop)
based on hand gestures.
 To provide a cost-effective solution using readily available components like
ESP32, DC motors, and L298N motor driver.
 To create an efficient and responsive gesture recognition system using only
a laptop camera.
 To enable wireless control of a robot without using external sensors.
 To integrate gesture interpretation with ESP32 for smooth and real-time
robot control.

The potential applications of such systems range from educational tools and
entertainment robots to industrial automation and assistive devices for the differently-
abled. This paper elaborates on the components, methodology, software
implementation, hardware design, and results of the developed gesture-controlled
robotic system.

2
1.2 Background and Motivation

The COVID-19 pandemic underscored the need for contactless and automated
systems, especially in healthcare facilities where the risk of infection is high. World
Health Organization (WHO) guidelines (2021) have encouraged the use of contact-
free technologies, including gesture and voice interfaces, to ensure safe interaction
between humans and machines in medical settings. Gesture-controlled robots provide
a critical solution to this challenge by reducing direct contact between healthcare
personnel and patients, especially in isolation wards and high-contamination zones.

Recent innovations in this domain have shown the feasibility of such systems. For
example, Ahmed et al. (2020) developed a real-time gesture recognition interface
using OpenCV for robotic control. Their system prioritized speed and simplicity over
heavy AI models, making it well-suited for embedded applications like this project.
Similarly, Raut & Deshmukh (2016) demonstrated gesture-based interfaces in low-
resource environments, further validating the applicability of vision-based systems in
embedded robotics.

Moreover, advanced systems like the Da Vinci Surgical Robot or Aethon’s TUG
robot (Gupta, JBHE 2021) are highly efficient but financially inaccessible for small
clinics or rural hospitals. There remains a large unmet need for affordable and
practical robotic systems that can perform essential tasks such as delivering food,
medicine, or documents in a clinical environment—exactly the niche this project aims
to address.

Our project, "Gesture Controlled Robot for Medical Assistance," aims to design and
develop a robot that can be controlled using real-time hand gestures detected by a
laptop’s inbuilt camera. The robot can perform movements such as forward,
backward, left, right, and stop, based on the gesture input. The robot's potential use is
in delivering essential items such as food, water, or medicines to patients in isolation,
minimizing the risk of infection for healthcare workers and enhancing the safety of
medical operations.

1.3 Objective

The primary objective of this project is to design and implement a gesture-controlled

robotic system specifically tailored for medical assistance applications. The robot is
3
expected to recognize hand gestures from a camera feed, interpret them through
image processing algorithms, and execute corresponding physical movements

Specific objectives include:

 To develop a gesture recognition module using a webcam and open-source

computer vision libraries such as OpenCV.
 To build a mobile robot platform using a microcontroller (such as Arduino or
Raspberry Pi) capable of receiving and executing commands via wireless
communication (Bluetooth or Wi-Fi).
 To establish real-time communication between the gesture interface (e.g., a
laptop or PC) and the robot to ensure seamless operation.
 To validate the system in a controlled environment such as a laboratory or
simulated hospital setting to demonstrate its functionality and effectiveness.
 To analyze the advantages, limitations, and future potential of gesture-
controlled systems in healthcare environments.
 The broader vision is to contribute to the development of assistive
technologies that empower healthcare workers and improve patient outcomes
through automation and intelligent interaction.

Fig.1.1 System Architecture

This figure illustrates the system architecture of the gesture-controlled robot. The
laptop, connected via USB, runs gesture recognition using a webcam and sends

4
commands to the ESP32 microcontroller. The ESP32 interprets these commands and
controls the L298N motor driver through four input pins (IN1–IN4). The L298N then
powers and drives two DC motors, enabling the robot's movement. A separate power
source supplies the required voltage to the motor driver for motor operation.

1.4 Problem Statement

 The healthcare industry, while technologically advanced in many respects, still

faces significant operational challenges, particularly in resource-constrained or
high-pressure environments. In hospitals and clinics, medical personnel are
often required to perform multiple tasks simultaneously, such as attending to
patients, managing records, and transporting supplies. During pandemics or
infectious disease outbreaks, the situation becomes even more critical, as
direct contact between staff and patients increases the risk of transmission.
 Current robotic solutions are often prohibitively expensive, complex to
operate, or unsuitable for the dynamic and cluttered environments typical of
healthcare settings. Moreover, most traditional robots rely on physical
interfaces (keyboards, joysticks, etc.) or verbal commands, which may not be
practical or hygienic in all situations.
 Hence, there is a need for a low-cost, easy-to-operate robotic system that
reduces physical contact, supports medical staff, and enhances operational
efficiency. A gesture-controlled robot can serve as an ideal solution, offering a
touchless and intuitive control mechanism suitable for use by personnel with
minimal technical training.

The central problem this project addresses is: "How can we design a reliable,
affordable, and user-friendly robot that responds to hand gestures to assist medical
personnel in routine tasks, while reducing physical contact and risk of contamination.

1.5 Scope of the Project

The scope of this project encompasses the complete lifecycle of developing a gesture-
controlled robot system, from requirement analysis to implementation and testing.
The robot is intended for indoor use within medical environments such as hospitals,
quarantine centers, clinics, and elderly care homes. While the prototype will

5
demonstrate basic movements, the design will be modular to allow for future
enhancements.

This project focuses on creating a working prototype of a gesture-controlled robot

that can serve in limited medical assistance tasks.

The scope includes

 Implementation of gesture recognition using Python and OpenCV

 Real-time control of robot using Wi-Fi communication.
 Movement capabilities suitable for flat surfaces and corridors.
 Use in environments like hospitals, quarantine centers, or even homes for
elderly care.

1.5.1 Key Features

[Link] Gesture Recognition Interface:

Using a webcam and a laptop, hand gestures will be detected in real-time using image
processing techniques. A predefined set of gestures (e.g., palm facing front for stop,
hand moving right for right turn) will be used to issue commands.

[Link] Robotic Platform:

The robot will be built on a wheeled chassis equipped with motors, sensors, and a
microcontroller unit (such as Raspberry Pi or Arduino). It will respond to directional
commands like forward, backward, left, right, and stop.

[Link] Wireless Communication:

A wireless communication protocol (Bluetooth or Wi-Fi) will be established between

the laptop and the robot to transmit gesture-derived commands instantly.

[Link] Basic Navigation:

The robot will be able to navigate simple hospital corridors or rooms without
advanced autonomous navigation or obstacle avoidance (although this could be added
later).

[Link] Use Case Demonstration:

A working demonstration will showcase how the robot can be used to carry
lightweight objects, such as medicine packets or documents, from one location to
another in a contactless manner.

6
1.5.2 Future Scope

The project’s future scope include:

 Advanced AI-based obstacle detection or autonomous path planning.

 Integration with hospital databases or IoT systems.
 Voice control or speech recognition interfaces.

The long-term vision is to use this foundation to create a multi-modal robot assistant
for smart healthcare environments that combines gesture control with autonomous
navigation, voice interface, and sensor-based obstacle avoidance.

1.6 Summary

The proposed gesture-controlled robotic system is composed of two primary

functional components, each playing a crucial role in achieving seamless, real-time
interaction between the user and the robot. These components are the Gesture
Recognition Unit and the Robot Movement Unit, which work in coordination to
interpret hand gestures and translate them into physical movements of the robot.

 Gesture Recognition Unit

At the heart of the system’s interface is a standard laptop equipped with a
built- in camera, which functions as the vision sensor. The laptop runs a
custom Python program developed using the OpenCV library, which is
responsible for processing real-time video input. The camera captures a
continuous stream of images or video frames of the user’s hand gestures.
These frames are analyzed using basic image processing techniques such as
color filtering, contour detection, and convex hull analysis, or possibly more
advanced methods like MediaPipe for gesture tracking. Once a gesture is
recognized, the program classifies it into predefined commands—such as
forward, backward, left, right, or stop—based on the hand’s orientation,
movement, or configuration.

 Robot Movement Unit

The second part of the system involves the robotic platform powered by an
ESP32 microcontroller, which is chosen for its integrated Wi-Fi capability and
compatibility with embedded robotics. The ESP32 acts as the receiver and
controller for movement commands transmitted from the laptop. Once a

7
gesture

8
is classified by the gesture recognition unit, the corresponding command is
sent wirelessly via Wi-Fi communication to the ESP32 on the robot. Upon
receiving the command, the ESP32 interprets it and activates the appropriate
motor controls using an L298N Motor Driver module. The motor driver, in
turn, regulates the motion of two DC motors mounted on a chassis, enabling
the robot to move in the desired direction.

9
 CHAPTER 2
LITERATURE REVIEW

2.1 Introduction
The integration of robotics and human-computer interaction has been a focus of
research for several decades. With the advent of affordable embedded systems,
machine learning, and computer vision techniques, gesture-based control systems
have become more practical and widespread. In healthcare, the need for contactless,
hygienic, and intuitive control mechanisms has led to an increasing interest in gesture-
controlled robotic systems.

This chapter explores existing research and technologies relevant to gesture

recognition, robotic movement, and medical robotics. The review aims to identify
gaps in current solutions and justify the need for a cost-effective gesture-controlled
robot designed for medical environments.

2.2 Gesture Recognition Techniques

Gesture recognition is a key component of human-machine interaction (HMI),
allowing machines to interpret human body movements, most commonly hand
gestures. Two primary approaches dominate the literature:

2.2.1 Sensor-Based Recognition

Sensor-based gesture recognition uses physical devices such as accelerometers,
gyroscopes, and flex sensors to capture hand and arm movements.

 “Design and Development of Gesture Controlled Robotic Arm” – IJCAT, 2023

Kumar and colleagues developed a flex-sensor based glove integrated with an
ESP32 microcontroller to control a robotic arm. The system demonstrated high
precision in joint manipulation, but required the user to wear a glove, which is less
desirable in medical environments due to hygiene and user comfort concerns.
 Wang and Song (2016) in their study on “IMU-Based Hand Gesture Recognition
for Industrial Applications” used inertial measurement units (IMUs) to detect
motion patterns for robotic control in manufacturing settings. While effective in
terms of motion accuracy and responsiveness, the setup was unsuitable for sterile
environments like operating rooms.

10
  “Gesture-Controlled Mecanum Wheel Car Using ESP32 and MPU6050,” JETIR,
2024. This research leveraged ESP32 with MPU6050 IMU sensors to achieve
omnidirectional movement through gesture recognition. While promising for
mobile robotics, it still relied on worn hardware, limiting its hygienic applicability.

Summary:
Sensor-based gesture recognition relies on physical, wearable hardware such as flex
sensors, accelerometers, and gyroscopes to capture movement data from the user’s
hands and arms. These systems are generally praised for their high accuracy, fast
response times, and minimal computational demands, making them suitable for
environments that require quick and precise interaction. However, their primary
drawback lies in the need for users to wear specialized equipment, which becomes a
significant limitation in medical contexts. In hospitals, especially in sterile
environments like operating rooms or isolation wards, hygiene and ease of use are
critical. The requirement for physical contact with devices—gloves, bands, or sensor
modules—raises concerns regarding infection control, maintenance, and user comfort.
While studies like those by Kumar et al. (2023) and Wang & Song (2016)
demonstrated effective control and responsiveness, they underscore the limited
practicality of such systems for clinical or contact-sensitive applications. The reliance
on body-worn gear, even when lightweight, does not align with the growing trend
toward contactless, intuitive interfaces in healthcare. In summary, while sensor-based
approaches excel in technical performance, they face key barriers to adoption in
medical settings due to hygiene concerns, discomfort, and setup complexity.
2.2.2 Vision-Based Recognition
Vision-based systems use cameras and software to detect and interpret gestures
without requiring any physical contact or devices on the user.

 “OpenCV Python: Real-time Hand Gesture Recognition,” Mallick & Kumar

(2019)
This work used OpenCV and Python to build a real-time gesture recognition
system. The system used thresholding, contour detection, and basic convex
hull algorithms to distinguish hand gestures. It emphasized simplicity and real-
time performance, making it suitable for low-cost systems like the one
proposed in this project.

11
  “Hand Gesture Recognition for Human-Computer Interaction,” S. S. Raut and
R. P. Deshmukh, IJSR, 2016. This study used a basic image processing
pipeline to detect static gestures using a camera. It helped validate the
feasibility of vision-based interfaces in constrained hardware environments,
although it lacked robustness to dynamic gestures or lighting variations.
 “Real-Time Remote-Controlled Human Manipulation Medical Robot Using
IoT,” Semantic Scholar, 2024. This study utilized real-time gesture recognition
using a webcam and MediaPipe, enabling an ESP32-based robot to be
controlled wirelessly. It focused on reducing latency and improving real-world
usability, making it highly relevant for real-time medical applications.
 Chen et al. (2019) – “Real-Time Hand Gesture Recognition Using CNNs,”
IEEE Access. This paper implemented convolutional neural networks (CNNs)
to classify hand gestures from video frames, achieving over 95% accuracy.
However, it required significant computational resources, making it more
suitable for advanced robotics rather than embedded systems.

Summary:
Vision-based gesture recognition systems use cameras and image processing
algorithms to detect hand gestures without any physical contact, offering a completely
touch-free interface. These systems are especially appealing in healthcare due to their
hygienic, user-friendly nature and the absence of wearable components. Basic
approaches, such as those demonstrated by Mallick & Kumar (2019) and Raut &
Deshmukh (2016), use OpenCV with techniques like thresholding, contour detection,
and convex hull algorithms to interpret gestures. These methods are lightweight, cost-
effective, and well-suited for real-time applications on resource-constrained platforms
such as Raspberry Pi or ESP32. More advanced systems integrate MediaPipe or
CNNs to improve gesture recognition accuracy and robustness under varying lighting
and dynamic conditions. For example, Chen et al. (2019) achieved high precision
using convolutional neural networks, though at the cost of increased computational
load, limiting its use in low-power embedded systems. Vision-based systems,
particularly those that balance accuracy with computational efficiency, offer a
compelling solution for real-time robotic control in healthcare. They support remote,
contactless operation, which aligns with global health recommendations emphasizing
non-invasive interfaces

12
post-COVID-19. Overall, vision-based approaches strike the best balance between
usability, hygiene, cost, and integration flexibility, especially for clinical applications.

2.3 Robotics in Healthcare

Robotic systems have been increasingly deployed in healthcare for surgery, logistics,
rehabilitation, and telepresence.

 “Medical Robots: Current Applications and Future Scope,” Gupta, JBHE,

2021 This review paper categorizes medical robots into surgical assistants,
delivery systems, and rehabilitation robots. The paper highlights devices like
the Da Vinci Surgical System and TUG delivery robot as high-precision but
expensive systems, often limited to well-funded hospitals.
 “Design and Analysis of Remote-Controlled Robotic Arms for Medical
Operation Purpose,” IARJSET, 2024. This paper introduced a cost-effective
robotic arm for remote surgical training, pointing out the importance of
modularity, reliability, and remote accessibility in medical robotics. It
reinforces the relevance of simplified control mechanisms like gesture
recognition in healthcare.
 WHO Guidelines on Contactless Technologies in Medical Care, 2021
The World Health Organization emphasized the importance of contactless
interfaces during the COVID-19 pandemic, citing gesture and voice interfaces
as key enablers for hygienic, non-invasive interaction.

Summary:
Robotic systems are becoming increasingly integral in healthcare, transforming how
medical services are delivered in surgical operations, patient logistics, rehabilitation,
and even remote presence. Modern healthcare robotics can be broadly categorized into
surgical robots, delivery robots, and assistive/rehabilitation robots, as discussed in
works like Gupta (2021). High-end systems like the Da Vinci Surgical Robot offer
extraordinary precision and minimally invasive surgery capabilities but are financially
inaccessible to many institutions, particularly in developing regions. Consequently,
there's a strong push for low-cost, modular, and user-friendly robotic platforms that
can be deployed in emergency, rural, or resource-constrained settings. Remote-
controlled and gesture-operated robots—like those discussed in the IARJSET (2024)
study—offer a viable alternative by enabling telemedicine and remote surgical

13
training. These

14
systems emphasize the need for simplicity in control mechanisms, adaptability, and
real-time performance. The COVID-19 pandemic accelerated the demand for
contactless technologies, with the World Health Organization (2021) advocating for
interfaces that reduce the risk of contamination. Gesture and voice interfaces were
especially recommended for their hands-free, user-centric control, making them ideal
for sterile environments. In essence, healthcare robotics is shifting from bulky, high-
cost installations to portable, affordable, and touch-free systems that enhance
accessibility and safety in diverse clinical environments.

2.4 Gesture-Controlled Robots

Research into gesture-controlled robots has evolved across both sensor-based and
vision-based implementations. Many studies have validated the concept but fall short
in terms of real-time performance, affordability, environmental adaptability, or ease
of use.

 “Gesture Controlled Robot Using Hand Gestures and Accelerometer,” P.

Yadav et al., IRJET, 2017. This early implementation used accelerometer
sensors and wireless communication to drive a simple robot. It validated the
potential of gesture control but required the user to wear a module, making it
less ideal for sterile or clinical environments.
 “IoT Based Hand Gesture Control Robot,” IJERT, 2023
The paper presented a robot that used hand gestures via a webcam and
OpenCV, processed on a Raspberry Pi, and controlled remotely. It balanced
cost, usability, and scalability, offering key insights into building real-time,
contactless systems.
 “A Real-Time Gesture Controlled Robot Car,” SpringerLink, 2025
This study utilized ESP32 and MediaPipe for accurate, vision-based gesture
recognition. It emphasized latency reduction and responsiveness, which are
critical for medical robotics in fast-paced environments.
 “Echo Glove: Assistive Smart Glove,” IJRASET, 2025
Though sensor-based, this study innovatively combined health monitoring
with gesture recognition in a smart glove, showing potential for
multifunctional assistive systems in hospitals.

15
Summary:
The field of gesture-controlled robots is experiencing rapid growth, with research
covering both sensor-driven and vision-based implementations. Early studies like
Yadav et al. (2017) showcased accelerometer-based robots that demonstrated the
feasibility of using hand gestures for control. However, these early systems relied
heavily on wearable hardware, which is increasingly viewed as a barrier in medical
and hygienic contexts. More recent developments, such as those described in the
IJERT (2023) and SpringerLink (2025) papers, focus on webcam-based gesture
recognition using OpenCV and MediaPipe frameworks, offering real-time
performance with no physical contact. These systems have proven capable of
accurately interpreting hand movements and relaying them wirelessly to robotic
platforms controlled by ESP32 or similar microcontrollers. Their emphasis on low-
latency communication, ease of use, and affordability makes them ideal candidates for
deployment in fast-paced medical environments. Other innovations, such as
EchoGlove (IJRASET, 2025), combine gesture sensing with health monitoring,
showing that multifunctionality in assistive technology is possible, though the
wearable nature still limits applicability in sterile settings. Collectively, the literature
illustrates that while gesture control is a practical and increasingly reliable interaction
method, success depends on adapting the design to the operating environment. For
healthcare, this means prioritizing contactless, lightweight, and low-cost solutions that
ensure both efficiency and hygiene.

2.5 Comparative Study of Existing Systems

1. Kumar et al. (2018) implemented a glove-based gesture control system using flex
sensors and an Arduino board. While functional, the major drawback of their system
is that it is not contactless, requiring the user to wear the glove for gesture input.

2. Ahmed et al. (2020) developed a vision-based gesture recognition system utilizing

OpenCV and a standard webcam. However, the system is limited by its short range,
making it ineffective for gestures performed at a distance.

3. Patel and Shah (2022) explored image processing applications for robotics using
OpenCV on a Raspberry Pi platform. Their project does not mention specific
limitations, implying a basic but functional implementation.

16
4. Shravani et al. (2020) designed a gesture-based wheelchair that assists mobility
through hand movements detected by an accelerometer. Despite its usefulness, the
system requires the user to wear a device, which may reduce comfort and ease of use.

5. Choudhury et al. (2019) created a remotely controlled medical assistance robot for
hospitals, built with Raspberry Pi and ultrasonic sensors. The primary limitations
include the high cost of hardware and limited automation capabilities.

6. Kumar et al. (2021) focused on IoT-based applications using ESP32 for real-time
device control over Wi-Fi. While effective for remote control, their solution does not
incorporate gesture recognition, limiting its intuitiveness.

7. Patil et al. (2018) worked on controlling a DC motor using an Arduino and L298N
driver to enable efficient motor direction control. Their work, however, was not
intended for gesture control or medical robotics, making it less relevant for healthcare
applications.

8. Singh et al. (2022) developed an advanced AI-based system capable of real-time

hand gesture recognition using CNN, OpenCV, and MediaPipe. Although highly
accurate, the system demands significant processing power, which may be impractical
for low- resource environments.

9. Ramesh et al. (2021) proposed a contactless delivery robot aimed at use in COVID-
19 hospital scenarios, using Arduino and Bluetooth for control. Their robot faced
limitations in communication range and exhibited delays in response time.

10. Deshmukh et al. (2019) built a hand gesture-controlled robot that operates through
a glove embedded with flex sensors, interfaced with an Arduino. The major limitation
is that it relies on physical wearable hardware rather than using a contactless camera-
based system.

11. Verma et al. (2023) investigated human-robot interaction in healthcare using a

combination of a laptop camera, Python, and ESP32. Although practical, the
performance of the system degrades significantly under low-light conditions.

12. Goyal et al. (2020) presented a sophisticated smart healthcare robotic system that
integrates AI, IoT, and sensor networks. Despite its advanced features, the solution is

17
highly complex and may be challenging to implement and maintain in typical clinical
settings.

13. Naik et al. (2021) developed a contactless robot for medicine delivery, using a
line- following approach controlled by an ESP32. However, the robot lacks gesture
input capabilities, reducing its interactivity.

14. Ahmed et al. (2022) enhanced gesture detection accuracy through real-time vision
tracking, employing MediaPipe and TensorFlow. The system achieves high
performance but is constrained by the need for substantial CPU and RAM resources.

15. Joshi et al. (2018) designed a simple motor control system using ESP32 and motor
drivers for real-time direction control. Although efficient, their system lacks a
graphical or gesture-based user interface, limiting ease of use.

16. Sharma et al. (2023) introduced a vision-based robot assistant that navigates based
on hand gestures, using Python, OpenCV, and a robotic chassis. The system, however,
suffers from latency in gesture detection, impacting its real-time performance.

17. Tanwar et al. (2020) implemented a Wi-Fi controlled vehicle using ESP32 and a
web interface. The robot operates only under manual control with no automation,
making it less suitable for hands-free or intelligent applications.

18. Mehta et al. (2022) built a robot for delivering PPE kits during the COVID-19
pandemic using Arduino and the L298N driver. Their design lacks advanced features
and follows a fixed path, which limits adaptability.

This Project aims to provide medical assistance using a low-cost, scalable system built
with a webcam, OpenCV, and wireless robot technology. Unlike many prior works, it
emphasizes affordability and ease of deployment while maintaining contactless
control capabilities.

18
S. STUDY/PROJECT TOPIC KEY TECHNOLOGY LIMITATIONS
NO. CONTRIBUTIONS USED
1 Kumar et al. (2018) Glove- Glove-based gesture Flex sensors, Not contactless
Based control Arduino
Gesture
Control
Using Flex
Sensors and
Arduino

2 Ahmed et al. (2020) Vision- Vision-based gesture OpenCV, Limited range

Based Hand input Webcam
Gesture
Recognition
Using
OpenCV
and
Webcam

3 Patel & Shah (2022) Raspberry Image processing OpenCV, No

Pi-Based interface Raspberry
Image Pi
Processing
for Gesture-
Controlled
Interfaces

4 Shravani et al., 2020 Gesture- Assisted mobility Accelerometer- Requires

based using hand gestures based gesture wearable device
Wheelchair control

5 Choudhury et al., Medical Remote-controlled Raspberry Pi, Costly hardware,

2019 Assistance robot for hospitals Ultrasonic limited
Robot sensors automation

19
6 Kumar et al., 2021 ESP32 in Real-time control Wi-Fi-based No gesture
IoT using ESP32 device control interface
Applications

7 Patil et al., 2018 DC Motor Efficient motor Arduino & Not focused on
Control via direction control L298N Driver medical robotics
L298N

8 Singh et al., 2022 AI-based Real-time hand CNN OpenCV High processing
Gesture gesture MediaPipe power required
Recognition interpretation

9 Ramesh et al., 2021 Robot for Designed robot for Bluetooth Limited range &
Contactless COVID hospitals Arduino delay issues
Delivery

10 Deshmukh et al., Hand Basic gesture bot Flex sensors, Not camera-
2019 Gesture with glove Arduino based
Controlled
Bot

11 Verma et al., 2023 Human- Gesture control in Laptop camera Weak

Robot healthcare robotics Python ESP32 performance in
Interaction low light

12 Goyal et al., 2020 Smart Integration of AI and ML, IoT & Very high
Healthcare robotics sensor networks system
Robotics complexity

13 Naik et al., 2021 Contactless Autonomous Line follower No gesture input

Medical medicine delivery ESP32
Robot

14 Ahmed et al., 2022 Real-Time Accurate gesture MediaPipe, High RAM/CPU

Vision detection with vision TensorFlow required
Tracking

15 Joshi et al., 2018 Motor Real-time direction ESP32, motor Lacks user
Control via control drivers interface
ESP32

20
16 Sharma et al., 2023 Vision- Gesture interface for Python OpenCV Latency in
based Robot navigation Robot Chassis gesture detection
Assistant

17 Tanwar et al., 2020 ESP32 Remote control ESP32, Web No automation,

Controlled with Wi-Fi Interface manual control
Vehicle

18 Mehta et al., 2022 COVID-19 Delivery of PPE kits Arduino & No advanced
Robot via robot L298N control, fixed
path

19 This Project Gesture Real-time control Webcam, Not used

Controlled using ESP32 OpenCV, for Wide
Robot For Wireless Robot Range
Medical
Assistance

Table 2.1: Literature study of Existing System

Table 2.2: Comparison Study

This comparison highlights the uniqueness of this project, which emphasizes
affordability, hygiene, wireless control, and medical usability within a real-time
operational setup.

21
2.6 Research Gaps and Motivation
While several studies have successfully implemented gesture-controlled robots, few
focus specifically on the medical context where hygiene, reliability, and usability are
critical. Moreover, many systems either rely on expensive hardware or require the
user to wear equipment—undesirable traits for hospital environments.

Identified research gaps include:

 Lack of fully contactless control systems in low-cost setups.

 Insufficient focus on real-time responsiveness using lightweight image
processing algorithms.
 Limited research on user-friendliness and training-free interfaces for non-
technical hospital staff.
 Absence of open-source designs that can be easily replicated and scaled in
developing nations.

This project is motivated by the urgent need for such a system that bridges the gap
between technical feasibility and practical usability in healthcare environments.

2.7 Summary

In summary, prior research establishes a strong foundation for gesture recognition and
robotic control but falls short in delivering a complete, hygienic, and affordable
solution for medical assistance. Vision-based gesture recognition using OpenCV
combined with a simple mobile robot platform offers a viable solution that meets
these needs.

This literature review confirms the relevance and importance of the proposed project,
which is designed to fill specific gaps by creating a cost-effective, gesture-driven
robotic assistant tailored for real-world medical use.

22
 CHAPTER 3
SYSTEM ANALYSIS AND
DESIGN
3.1 Introduction
System analysis and design form the backbone of any engineering project, as they
determine the structure, behavior, and interactions among various components of the
system. For a gesture-controlled robot for medical assistance, it is crucial to identify
user needs, analyze the problem, and design a system that is not only technically
feasible but also functionally effective within a healthcare setting.

This chapter details the analytical foundation and architectural layout of the proposed
system. It outlines the requirements specification, system block diagram, detailed
module description, flow of control, and hardware-software interaction, providing a
comprehensive blueprint for implementation.

3.2 Requirements Analysis

The system requirements are categorized into functional and non-functional to clearly
define what the system should do and how it should behave.

3.2.1 Functional Requirements

 Gesture Capture: The system must detect and track hand gestures in real-
time using a webcam.
 Gesture Recognition: Recognized gestures should be mapped to control
commands such as forward, backward, left, right, and stop.
 Command Transmission: The commands should be transmitted wirelessly
to the robot using Bluetooth/Wi-Fi.
 Movement Execution: The robot should accurately interpret and execute
commands on a wheeled platform.
 System Feedback: The robot should provide visible or audible confirmation
(e.g., LED blink or buzzer) for successful command reception.

3.2.2 Non-Functional Requirements

 Real-Time Response: System latency from gesture input to robot response

must be under 1 second.

23
  Safety and Reliability: The system should avoid erratic motion or crashes;
basic timeouts or error checks should be implemented.
 Usability: The interface should be intuitive enough for non-technical users
(e.g., hospital staff).
 Portability: The robot and gesture detection system should be portable and
easily deployable in different rooms.
 Cost-Effectiveness: The system must be constructed with affordable
components without sacrificing critical functionality.

3.3 System Architecture

The system is composed of two primary subsystems:

● Gesture Control Interface (hosted on a laptop/PC):

Camera captures real-time video feed.

OpenCV-based image processing identifies hand gestures.

Recognized gestures are mapped to directional

commands. Wireless transmission of commands via

Bluetooth/Wi-Fi.

● Robot Movement Platform:

Microcontroller (e.g., Arduino UNO or Raspberry Pi) receives commands. Motor

driver circuit controls the movement of DC motors.

Power supply manages robot operations. Optionally,

LEDs or buzzers provide user feedback.

The block diagram shown in Fig 3.1 in the next page, visually represents the
interaction between the Gesture Control Interface and the Robot Movement Platform.
It outlines the flow of data from gesture detection to wireless transmission and finally
to robot actuation, illustrating how each subsystem contributes to the overall
operation.

24
3.3.1 Block Diagram

Fig 3.1 : Block Diagram of System Architecture

3.4 Detailed Module Description

3.4.1 Gesture Capture Module

Uses a webcam to capture a real-time video stream.

Operates at ~30 frames per second (FPS) to ensure smooth tracking.

Focused on isolating hand region using color segmentation (e.g., HSV color filtering)
or background subtraction.

3.4.2 Gesture Recognition Module

Extracts features such as hand contour, finger count, and orientation.

Uses predefined templates or logic (e.g., number of fingers shown) to map to actions.

Implemented using OpenCV with optional use of ML-based classifiers if needed for
complex recognition.

3.4.3 Command Transmission Module

● Based on PySerial (for Bluetooth) or socket programming (for Wi-Fi).

● Translates recognized gestures into ASCII commands (e.g., 'F' for forward).
● Sends data wirelessly to the robot microcontroller.

3.4.4 Robot Control Module

● Receives command via serial communication.

25
 ● Arduino processes command and uses a motor driver to actuate wheel
movement.
● Capable of interpreting:

'F' = Forward
'B' = Backward
'L' = Left Turn
'R' = Right Turn
'S' = Stop

Table 3.1 :Control Module

3.4.5 Power Management

Separate regulated power supply (e.g., 12V battery) used to power motors. Voltage

regulators (7805 IC) manage current for microcontroller and peripherals.

3.5 System Flow

The system follows a sequential and modular control flow:

● User shows hand gesture in front of the webcam.

● Camera captures video and sends frames to the gesture recognition module.
● System analyzes gesture, maps it to a command.
● Command is transmitted wirelessly to the robot.
● Microcontroller receives command and actuates motors accordingly.
● Robot performs movement, potentially delivering a payload.

A simplified flowchart of this operation can be illustrated as

follows:

26
 Fig 3.2 :Flowchart of Proposed Method

3.6 Hardware-Software Interaction

This section explains how software modules interface with hardware:

● Python and OpenCV handle video processing and gesture interpretation on the
host system.
● PySerial / Socket Communication is used for sending data to the robot.
● Arduino Sketch written in C++ receives serial input and maps characters to
GPIO pin toggling for motor control.
● L298N Motor Driver acts as the interface between logic-level microcontroller
signals and high-current motors.

Fig 3.3 :Hardware Software Interaction Through ESP-32

27
3.7 Challenges and Design Considerations

Lighting Conditions: Varying brightness can affect gesture recognition accuracy;

robust thresholding and filtering techniques are necessary.

Wireless Delay: Bluetooth has limited range and may cause delays; Wi-Fi modules
like ESP8266 can be considered for more scalable communication.

Battery Life: Continuous motion drains batteries quickly; power management

strategies like sleep modes and efficient motor usage are essential.

Gesture Complexity: Overly complex gestures increase error rate; a minimal and
intuitive gesture set is recommended.

3.8 Summary

This chapter laid the groundwork for implementing the gesture-controlled robot by
analyzing system requirements and presenting a structured design approach. With a
clear modular architecture, defined data flow, and hardware-software interfacing
strategies, the project is well-positioned for successful development and deployment
in medical assistance environments.

28
 CHAPTER 4
Implementation and testing

4.1 Hardware Implementation

The hardware implementation involves integrating components such as ESP32, laptop
with camera, L298N motor driver, DC motors, robot chassis, jumper wires, and
power source. Each part plays a critical role in ensuring the smooth operation of the
gesture- controlled robot.

Fig 4.1 : Components for Hardware Implementation

4.1.1 Laptop with Inbuilt Camera

The laptop acts as the main control unit for gesture recognition. The inbuilt webcam
continuously captures live video, which is processed using Python and OpenCV. The
detected gestures are translated into commands and sent to the ESP32 over serial or
wireless communication protocols such as Wi-Fi or Bluetooth.

4.1.2 ESP32 Microcontroller

Fig 4.2: Esp-32 microcontroller

29
The ESP32 is a powerful microcontroller with inbuilt Wi-Fi and Bluetooth modules.
It is responsible for receiving gesture-based commands from the laptop and
translating them into appropriate motor actions. The ESP32 is programmed using the
Arduino IDE with libraries compatible for motor control and serial
communication. Let’s figure out why we are using ESP 32 over other microcontroller.

Feature/ ESP32 Arduino STM32 (e.g. Blue Pill) Raspberry NodeMCU

Spec Uno Pi Pico (ESP8266)
(ATmega
328P)
Process Dual- 8-bit AVR 32-bit ARM Cortex- Dual-core Single-core
or core @ 16 M3 @ 72 MHz ARM @ 80/160
Xtensa MHz Cortex- MHz
@ 240 M0+ @133
MHz MHz
RAM ~520 2 KB 20 KB 264 KB 160 KB
KB
SRAM
+
PSRAM
options
Flash 4MB 32 KB 64 KB 2MB 4MB
(externa external
l)
GPIO ~30 14 ~37 26 17
Pins
Wi-Fi Built-in External External module (via external Built-in
module needed module)
needed
Bluetoo (Classic (unless STM32WB
th + BLE) series)
Camera (ESP32-
Support CAM
version)
USB UART Built-in (via USB) Built-in Built-in
Support to USB USB via
(via ATmega1
CP2102 6U2
)

30
 Power Good Moderate Good (low-power Good Moderate
Efficien (Deep modes)
cy Sleep
modes)
Price ₹300– ₹350– ₹200–₹300 ₹350–₹450 ₹250–₹350
(approx. ₹400 ₹500
)
Progra Arduino Arduino STM32Cube, Arduino Arduino Arduino
mming IDE, IDE IDE IDE, IDE, Lua
MicroPy MicroPytho
thon, n
ESP-
IDF
Use Wi- Entry- High-performance Affordable IoT tasks
Case Fi/Bluet level control apps MCU tasks with Wi-Fi
Fit ooth + robotics
Real-
time
control
Table 4.1 : Comparison of ESP 32 with other microcontrollers

Fig4.3: Pin configuration of ESP-32

1. Built-in Wi-Fi and Bluetooth

ESP32 has integrated Wi-Fi and Bluetooth, enabling wireless gesture control (e.g., via
smartphone, web interface, or Bluetooth module) without needing external modules
like HC-05 or ESP8266.

2. High Processing Power

31
 With a dual-core 32-bit processor and clock speeds up to 240 MHz, ESP32 can
efficiently process real-time sensor data (from accelerometers, gyroscopes, etc.)
without lag.

3. More GPIO and Peripheral Support

It supports I2C, SPI, UART, PWM, ADC, DAC, and capacitive touch, which are
useful for connecting multiple sensors and actuators for gesture input and robot
control.

4. Low Power Consumption

ESP32 supports deep sleep modes, making it suitable for battery-powered gesture-
controlled robots, especially wearables or handheld units.

5. Cost-effective

Compared to microcontrollers with similar wireless features (e.g., STM32 with

external comm modules), ESP32 provides a powerful and affordable all-in-one
solution.

6. Large Community & Library Support

ESP32 is widely supported by Arduino IDE, PlatformIO, and MicroPython, with

extensive libraries for gesture recognition, wireless communication, and robotics.

Summary:

ESP32 is chosen because it offers wireless communication, real-time processing,

versatile I/O, and energy efficiency, all at a low cost—making it ideal for modern
gesture-controlled robots.

32
4.1.3 L298N Motor Driver Module

Fig 4.4: L298N Motor Driver Module

This module acts as an interface between the ESP32 and the DC motors. It amplifies
the low-power control signals from the ESP32 to drive the motors with sufficient
current and voltage. The L298N allows control of the robot's direction and speed
through PWM (Pulse Width Modulation) signals.

4.1.4 DC Motors

Fig 4.5: L293 D Motor

Two geared DC motors are used to provide locomotion. These motors are mounted on
a lightweight chassis and are driven independently for differential steering. Motor
specifications were selected to balance speed and torque suitable for indoor

33
navigation.

34
4.1.5 Robot Chassis with Wheels

A predesigned two-wheel drive (2WD) chassis made of plastic/acrylic is used. It

provides mounting space for motors, wiring, and the power source. The wheels are
coupled to the DC motors using compatible hubs, ensuring stable movement across
flat surfaces.

4.1.6 Jumper Wires

Jumper wires are used for flexible connections between the ESP32, motor driver, and
other components. Careful wiring is done to ensure minimal resistance and signal loss
during transmission.

4.1.7 Power Source

The power supply consists of a 3-cell battery pack (typically 11.1V) used to drive the
motors and power the L298N module. A separate voltage regulator is used to supply
5V to the ESP32, protecting it from overvoltage damage.

Fig 4.6 (a) : Hardware Implementation (Top View)

Top View Description

The top view of the gesture-controlled robot provides a clear and complete layout of
the hardware components and their assembly. The chassis used is a sturdy double-

35
layered red acrylic platform, suitable for compact robotics applications. On top of the
chassis, we can see:

 Four yellow wheels (two on each side) connected to DC motors, allowing for
differential drive movement.
 ESP32 development board mounted on a perforated PCB (protoboard) near the
front side of the chassis. This board acts as the microcontroller unit that
processes gesture commands received from the laptop.
 L298N motor driver module, which is responsible for driving the motors based
on signals from the ESP32. It is positioned in the middle, making wiring
convenient.
 Jumper wires connecting the ESP32 and L298N module to the motors and
power supply.
 Battery holder at the rear end, housing two 18650 Li-ion cells (7.4V), which
power the motor driver and ESP32 board.

The wiring is neatly routed, ensuring compactness while avoiding entanglement

during motion. The view also highlights proper component spacing for heat
dissipation and accessibility.

Fig 4.6 (b) : Hardware Implementation (Side View)

36
Side/Angular View Description

The side or angular view of the robot helps visualize the vertical arrangement and
depth of hardware integration:

 The ground clearance and wheel placement show the robot's ability to navigate
over small obstacles and uneven surfaces, ideal for indoor medical use.
 The ESP32 module and motor driver are securely elevated using nuts and
spacers on the base chassis, protecting them from vibration and surface
contact.
 From this view, the battery alignment and its compact fit in the holder are
clearly visible, ensuring the center of gravity remains low for stability.
 The yellow gear motor housing is prominent, showing how each motor is
directly mounted on the chassis using screws, allowing smooth torque transfer
to the wheels.
 The use of color-coded wiring simplifies debugging and improves the
aesthetics of the overall system.

This perspective provides a good understanding of the mechanical robustness,

symmetry, and wiring clarity, all crucial for effective movement and gesture-
controlled navigation.

4.2 Software Implementation

The software system consists of two main parts: gesture detection and wireless
command transmission.

4.2.1 Gesture Recognition Using OpenCV

Python scripts using OpenCV are deployed on the laptop. The following operations
are performed:

Video Capture: Continuous stream acquisition from the webcam.

Background Subtraction & Filtering: Helps isolate hand gestures.

Contour Detection: Detects fingers based on the shape of the

hand.

37
Gesture Classification: Maps gestures (e.g., fist, palm, pointing) to predefined
commands such as forward, backward, left, right, and stop.

38
 Gesture Recognition Using OpenCV

The gesture recognition system is developed using Python with the OpenCV and
MediaPipe libraries. OpenCV (Open Source Computer Vision) is used for real-time
video capture and image processing, while MediaPipe is employed to detect and track
hand landmarks efficiently.

The process begins with continuous video stream acquisition from the laptop's inbuilt
webcam. Each frame is processed to detect a human hand and extract key landmarks
such as fingertips and joints. MediaPipe's hand-tracking module provides 21
landmark points, which are used to determine hand orientation and gesture shape.

Predefined gestures such as an open palm, closed fist, or directional finger pointing
are mapped to specific robot movement commands — for example:

To reduce false detections and improve stability, frame smoothing and gesture
confirmation over multiple frames are applied before transmitting a command. Once
the gesture is recognized, the corresponding control signal is sent to the ESP32
microcontroller over serial communication.

This vision-based approach allows for intuitive, touch-free control without the need
for external sensors or wearable devices.

 Command : Forward Movement

Gesture Description:

Hand held straight with all fingers extended, palm facing directly toward the camera.
Function:
Commands the robot to move forward in a straight path.

39
 Fig 4.7(a) :- Forward Command Recognition

 Command : Stop the Movement

Gesture Description:
Open palm raised vertically like a “halt” signal, with fingers spread wide.
Function:
Immediately stops the robot’s movement for safety or control.

Fig 4.7(b) :-Stop Command Recognition

40
 Command : Back Movement
Gesture Description:
Closed fist (all fingers curled in), held in front of the camera.
Function:
Commands the robot to move backward in reverse motion.

Fig 4.7(c) :-Back Command Recognition

 Command : Left Movement

Gesture Description:
The robot rotates to its left when the user shows an open left hand, with fingers
splayed and palm facing the camera.
Function:
Initiates an anti-clockwise rotation, used to turn or align the robot to a new direction.

41
 Fig 4.7 (d):-Left Command Recognition
 Command : Right Movement
Gesture Description:
Right hand extended outward, fingers spread, palm facing the camera.
Function:
Commands the robot to rotate clockwise (right turn in place).

Fig 4.7 (e) :- Right Command Recognition

4.2.2 Communication with ESP32

The commands generated by the gesture recognition system are transmitted to the
ESP32 via Bluetooth (Serial over Bluetooth Classic or BLE). Python's pyserial or
bleak library is used to handle communication.

42
4.2.3 ESP32 Control Code

ESP32 receives characters (like ‘F’, ‘B’, ‘L’, ‘R’, ‘S’) over its serial interface and
drives the L298N module accordingly. The ESP32 code uses digital pins to control
the direction of the DC motors based on received commands.

4.3 Integration and Testing

4.3.1 Module Testing

Each component was tested independently:

● Camera Module: Verified frame rate and image clarity.

● ESP32 Communication: Checked Bluetooth pairing and data reception.
● Motor Driver & Motors: Tested for directional accuracy using manual inputs.

4.3.2 End-to-End Testing

An integrated test was conducted where gestures detected on the laptop were used to
drive the robot. The communication, latency, accuracy of movement, and gesture
recognition reliability were evaluated under different environmental conditions.

4.3.3 Test Results

Test Gesture Command Robot Action Status

T1 Open Palm Stop (S) Robot stopped Pass

T2 Point Up Forward (F) Moved forward Pass

T3 Point Down Backward (B) Reversed Pass

T4 Point Left Left (L) Turned left Pass

T5 Point Right Right (R) Turned right Pass

Table 4.2 : Test result

4.4 Challenges and Solutions

Bluetooth Pairing Instability: Replaced with Wi-Fi UDP communication as a fallback.

Hand Gesture Confusion: Added temporal smoothing to confirm gesture persistence

before sending commands.

43
Motor Overload Issues: Calibrated PWM signals and used current-limiting resistors
for safety.

4.5 Final Implementation Outcome

The final robot responded to gesture commands with a delay under 300ms. The
ESP32 handled serial communication and motor actuation effectively. The robot
successfully demonstrated basic autonomous navigation controlled by hand gestures,
making it a suitable prototype for non-contact operation in medical environments.

44
 CHAPTER 5
RESULT AND DISCUSSION

5.1 Overview of Testing Process

After completing the hardware integration and software development, the robot was
tested under various controlled and semi-controlled environments to evaluate:

 Responsiveness to hand gestures.

 Accuracy of gesture recognition.
 Stability of wireless communication (Bluetooth/Wi-Fi).
 Mobility and obstacle-handling performance.
 Suitability for use in basic medical assistance scenarios.

5.2 Evaluation Parameters

The evaluation focused on several qualitative and quantitative parameters:

PARAMETER DESCRIPTION

Gesture Recognition How accurately gestures were identified using

Accuracy the camera and OpenCV
Communication Delay Time delay between gesture execution and robot
response

Motor Control Precision Whether the robot moved as per expected path based
on gesture

Stability and Power Whether the robot could operate for sustained
periods without reboot/failure

Safety Safety measures to prevent random motion or

electrical overload

Usability in Medical Context How effectively the robot could perform non-contact
motion in a clinical setup

Table 5.1: Parameters for Testing

5.3 Test Environment Setup

● Surface: Smooth indoor tile flooring.
● Lighting: Moderate indoor lighting (~300 lux).

45
 ● Testing Range: Gesture control within 2 meters of the robot.
● Robot Load: Empty chassis, approximating typical lightweight utility (like
carrying medicine or documents).
● Power Supply: 11.1V Li-ion battery pack for motor driver; laptop and ESP32
powered independently.

5.4 Performance Results

5.4.1 Gesture Recognition Accuracy

Using OpenCV with basic preprocessing and contour detection, the system achieved a
gesture recognition accuracy of approximately 92% in consistent lighting.
Performance degraded slightly under shadow-heavy or low-light environments.

GESTURE CORRECT DETECTIONSMISCLASSIFICATIONS (%)

(%)

Palm (Stop) 96% 4%

Fist (Forward) 91% 9%

Left Hand 89% 11%

Right Hand 93% 7%

Down Palm 90% 10%

Table 5.2 : Gesture Accuracy

5.4.2 Command Latency

The average time from gesture execution to robot response was measured across 100
trials.

1. Average latency: 250–300 ms

2. Maximum latency (under load): 400 ms
3. Minimum latency: 180 ms

This is acceptable for non-critical medical applications (such as delivering medicines

or notes).

46
5.4.3 Robot Mobility and Control

The ESP32 interfaced with the L298N motor driver successfully, and motor
commands were executed with precision.

1. Forward/Backward Movement: Smooth and straight.

2. Turning Radius: ~25 cm.
3. Motor Speed Control: Fixed speed (no PWM variation implemented to simplify
design).

The chassis moved reliably on tiled floors with enough grip and power.

5.4.4 Connectivity Stability

Bluetooth Mode: Occasional dropouts in presence of electromagnetic interference or

when used beyond 3 meters.

Wi-Fi Mode (optional): More stable but required local network configuration.

Recommendation: Use Bluetooth for simplicity in small areas; upgrade to Wi-Fi for
scalability.

5.5 Limitations Observed

Despite achieving the intended objectives, the following limitations were noted:

● Lighting Sensitivity: Gesture recognition heavily depends on ambient light.

Sudden changes in light could affect accuracy.
● Gesture Confusion: Similar-looking hand gestures caused misinterpretations
occasionally.
● Obstacle Detection: The robot cannot detect or avoid obstacles autonomously; it
assumes a clear path.
● Power Limitations: The motors and ESP32 rely on separate power; incorrect
voltage caused inconsistent performance.
● Scalability: Currently single-user and line-of-sight operation; not designed for
complex navigation.

5.6 Discussion

The system demonstrated a viable and practical application of computer vision and
microcontroller-based robotics. Its real-time response and non-contact control design

47
make it suitable for basic assistance tasks in medical environments, particularly where
hygiene and social distancing are crucial.

From a broader perspective, this project also highlights the potential for low-cost,
customizable assistive robotics in under-resourced healthcare facilities. The
integration of hand gestures for robot navigation aligns with current trends in intuitive
human- robot interaction (HRI).

Further enhancements such as adding ultrasonic sensors for obstacle avoidance,

enhancing gesture detection with machine learning, or integrating speech recognition
could significantly elevate its utility.

The developed system effectively demonstrates a feasible and functional application

of computer vision integrated with microcontroller-based robotics for hands-free
navigation. The robot responds in real-time to hand gesture inputs, allowing it to
move in specific directions without any physical contact. This non-contact control
mechanism is especially valuable in medical and healthcare settings, where
maintaining hygiene and minimizing surface contact are paramount — such as in
isolation wards, ICUs, and during post-operative care where manual intervention must
be minimized.

One of the key highlights of this project is its practical utility in real-world scenarios,
particularly in the post-pandemic era, where minimizing human touchpoints in
clinical environments has become a standard safety protocol. The gesture recognition
interface, implemented via the laptop's inbuilt camera, eliminates the need for
additional sensors or wearable devices, making the solution more accessible and cost-
effective.

From a broader perspective, the project successfully illustrates the potential of low-
cost assistive robotic systems tailored for under-resourced hospitals and clinics. The
core components—ESP32 microcontroller, L298N motor driver, and basic computer
vision—are inexpensive and widely available, making replication or further
development of the system feasible even in developing regions. Additionally, the
robot's mechanical design is simple yet functional, with a four-wheel drive system
ensuring stability and ease of movement across typical indoor flooring.

 Machine learning models could be deployed to enhance gesture recognition

accuracy, reduce false positives, and adapt to different users’ hand shapes and
48
lighting conditions.

49
 Voice command or speech recognition modules could serve as alternative input
methods, offering additional flexibility and accessibility, especially for users with
limited hand mobility.
 A cloud-connected platform could be added for remote monitoring, control, and
data logging, facilitating deployment in telehealth scenarios.

In conclusion, the gesture-controlled robot not only fulfills its intended objective of
providing basic medical assistance through gesture-based navigation but also serves
as a foundation for future development of intelligent, contactless, and autonomous
medical robotics. It successfully bridges simplicity, affordability, and innovation—
three essential pillars for real-world adoption in healthcare environments.

50
 CONCLUSION AND FUTURE SCOPE
The project titled “Gesture Controlled Robot for Medical Assistance using ESP32”
was developed with the objective of creating a cost-effective, contactless, and
intuitive system to aid patients and healthcare staff in hospitals. The central idea was
to use hand gestures as input commands captured through a laptop camera, which are
then processed and transmitted wirelessly to a robotic platform powered by ESP32.
This robot responds in real-time, executing motion commands (forward, backward,
left, right) based on the user's gestures.

The entire system architecture integrates hardware components including the ESP32
microcontroller, L298N motor driver, DC motors, and robot chassis, with software
tools such as Python, OpenCV, and MediaPipe for gesture detection. The use of Wi-Fi
communication via ESP32 enabled real-time, wireless control, eliminating the need
for wired connections and offering greater mobility and convenience.

This project was inspired by the need for contactless interaction in medical
environments, especially in contexts like the COVID-19 pandemic, where reducing
physical touch became vital for infection control. Gesture-based control adds a layer
of convenience for patients with limited mobility, allowing them to interact with
assistance systems in a non-verbal and non-tactile manner.

6.1 Key Outcomes and Achievements

✅ Successful Real-Time Gesture Detection:

The implementation of gesture recognition using OpenCV and MediaPipe achieved

high accuracy in detecting and interpreting standard hand gestures.

✅ Wireless Control using ESP32:

The ESP32 microcontroller effectively received gesture commands over Wi-Fi,

processed them, and drove the robot accordingly via the motor driver.

✅ Low-Cost and Modular Design:

51
The system was developed using commonly available and low-cost components,
making it scalable and replicable for educational or healthcare institutions.

✅ Contactless Operation:

The robot responds to visual gestures without requiring any wearable devices or
physical contact, aligning with hygiene protocols in medical environments.

✅ User-Friendly Interface:

The system operates via a simple camera interface and hand movements, which do not
require technical knowledge, making it suitable for both patients and healthcare staff.

6.2 Limitations of the Project

Despite the successful implementation, several limitations were observed:

❌ Limited Range and Frame Rate:

The performance of gesture detection can degrade in poor lighting conditions or at

lower camera frame rates.

❌ Line-of-Sight Dependency:

Since the system uses a camera for input, gestures must remain within the field of
view. Obstructed or fast gestures can be missed.

❌ No Obstacle Avoidance: The current robot does not incorporate obstacle sensors,
which poses safety risks in real-world medical environments.

❌ Limited Gesture Vocabulary: Only basic directional gestures were implemented. A

more extensive set of commands could improve usability.

❌ Lack of Feedback Mechanism: The system does not currently provide visual or
audio feedback to confirm command reception, which may confuse first-time users.

52
6.3 Future Scope

The development of a gesture-controlled robot for medical assistance using ESP32

represents a significant step toward enhancing healthcare delivery through automation
and touchless interaction. However, the potential for improvement and expansion
remains vast. Several future directions can be explored to enhance the functionality,
adaptability, and intelligence of the system.

6.3.1 Integration with AI and Machine Learning

In future iterations, machine learning algorithms can be integrated to improve gesture

recognition accuracy and adaptability to different users. Using neural networks like
Convolutional Neural Networks (CNNs) or frameworks such as TensorFlow and
MediaPipe can enable the system to recognize a wider range of complex gestures,
adapt to different hand sizes or skin tones, and improve performance in varied
lighting conditions. These advancements can make the robot more reliable and
context-aware in real-world medical environments.

6.3.2 Incorporation of Voice Commands and Multimodal Interaction

To make the system more inclusive and user-friendly, future versions can combine
gesture recognition with voice commands, eye-tracking, or facial recognition. This
multimodal control approach will enhance accessibility for users with temporary or
permanent disabilities and allow for more nuanced interactions. For example, voice
commands can be used to confirm gestures or provide backup instructions when
gesture detection fails.

6.3.3 Autonomous Navigation and Obstacle Avoidance

Currently, the system requires manual or gesture-based directional control. Future

improvements can include autonomous navigation features using sensors such as
LiDAR, ultrasonic sensors, or computer vision. This would allow the robot to
navigate hospital corridors or patient rooms autonomously while avoiding obstacles
and dynamically adjusting its path. Integration with SLAM (Simultaneous
Localization and Mapping) algorithms could further enhance autonomous mobility.

6.3.4 Expansion into Telemedicine and Remote Operations

The system can be further enhanced to support telemedicine applications. By enabling

remote gesture control over the internet (e.g., via MQTT or cloud-based platforms),

53
healthcare professionals could guide or operate the robot from a remote location. This
can be highly beneficial in rural or quarantine zones where physical presence is
restricted. Cameras, microphones, and sensors on the robot can transmit real-time data
to doctors, facilitating remote diagnostics and care.

6.3.5 Use of Advanced Sensors and Embedded Systems

Future systems can incorporate high-precision IMU sensors, depth cameras (such as
Intel RealSense), or wearable gesture devices (e.g., smart gloves) to capture gestures
more accurately. In addition, more powerful embedded platforms such as the
NVIDIA Jetson Nano or Raspberry Pi 5 can be used in place of the ESP32 when high
computational power is needed, especially for real-time image processing and AI
model inference.

6.3.6 Battery Optimization and Wireless Charging

Current systems often depend on wired or limited-capacity batteries. Future work

could involve optimizing battery usage through better power management algorithms
and exploring wireless charging options. This enhancement would extend the robot’s
operational time and reduce downtime in medical facilities.

6.3.7 Enhanced Safety and Compliance Features

For the robot to be used in real hospital environments, it must comply with medical-
grade safety standards (e.g., ISO 13485 or IEC 60601). Future designs can
incorporate anti-collision features, patient safety measures, fail-safes, and hygienic
enclosures that make them safe and cleanable in clinical settings. Additional sensors
can be used to detect patient proximity and stop the robot automatically.

6.3.8 Integration with Hospital Management Systems (HMS)

In the future, gesture-controlled robots can be linked with hospital management

software to enable automated delivery of medical items, patient monitoring, and
location tracking. For instance, a nurse could simply gesture to call the robot to
deliver a medication tray from the pharmacy or lab reports from diagnostics to the
concerned ward.

6.3.9 Scalability for Mass Deployment

To support large-scale implementation, future work should address cost-effective

hardware designs, modular construction, and customizable software interfaces.
Cloud-

54
based updates and remote diagnostics can be incorporated to allow easy maintenance
and upgrades across hospital networks.

6.3.10 Applications Beyond Healthcare

While this project focuses on medical assistance, the underlying technology can be
repurposed for other domains such as elderly care, industrial automation, smart
homes, and education. Gesture-controlled robotic systems can assist in any context
where contactless interaction, remote control, or automation is valuable.

6.4 Future Scope (Extended with Advanced Technologies)

As gesture-controlled medical robots evolve, integration with advanced perception

and communication technologies is essential to meet the growing complexity of
hospital environments and medical tasks. Below are extended future directions
incorporating state-of-the-art modules such as CAN cards, GPS, LiDAR systems, and
camera-based mapping.

6.4.1 CAN Bus Integration for Modular Communication

Integrating a Controller Area Network (CAN) card will facilitate robust, real-time
communication between different robot components such as motor drivers, sensors,
and safety modules. This will:

 Improve reliability in signal transmission.

 Enable scalable multi-module communication (e.g., for multi-DOF robotic
arms).
 Ensure system robustness against electromagnetic interference in hospital
environments.

6.4.2 GPS for Outdoor Medical Logistics

Although indoor environments are the primary application, GPS modules can be
useful for outdoor scenarios such as:
 Transporting medical supplies between buildings in a hospital campus.
 Managing emergency relief logistics in disaster zones. In
such cases, combining GPS with real-time kinematic (RTK) corrections can
ensure high-accuracy outdoor navigation.

55
 6.4.3 Navigation LiDAR for Autonomous Indoor Movement
Adding a navigation LiDAR (e.g., 2D/3D SLAM-capable units) will allow the robot to:

 Build accurate indoor maps of hospital layouts.

 Navigate autonomously and efficiently even in dynamic environments.
 Avoid static and moving obstacles using real-time sensor fusion.

This enhances the system's intelligence, moving it beyond gesture-based direction

control to full autonomous operation.

6.4.4 Safety LiDAR for Human Detection and Collision Avoidance

To ensure patient and personnel safety in medical spaces, safety-rated LiDAR systems
(e.g., Sick microScan3 or Hokuyo URG) can be added. These provide:

 Proximity alerts and emergency stops.

 Human recognition in the robot’s path.
 Multi-zone safety control (warning, slow, and stop zones).

Such features are essential for certification under health and safety regulations for
medical robotics.

6.4.5 High-Resolution Camera Systems for Enhanced Gesture and Scene

Recognition

Advanced RGB-D cameras (like Intel RealSense or ZED) can improve both gesture
recognition and environmental perception by providing:

 Depth sensing for 3D gesture analysis.

 Facial recognition for personalized responses.
 Scene understanding for dynamic decision-making (e.g., detecting a blocked
hallway).

6.4.6 Simultaneous Localization and Mapping (SLAM)

Integration of SLAM algorithms with LiDAR and camera inputs will allow the robot
to:

56
 Create real-time maps of its environment.
 Localize itself accurately without external beacons.
 Continuously update its path plan in response to changes (e.g., new beds, staff
movement).

This feature is especially important in dynamic environments like emergency wards

or ICUs.

6.4.7 Edge Computing for Real-Time Decision Making

Processing camera and LiDAR data on edge devices such as NVIDIA Jetson Nano,
Xavier, or Raspberry Pi 5 with AI accelerators allows:

 Onboard processing of complex gestures.

 Real-time object recognition and path planning.
 Reduced latency compared to cloud-based processing.

This improves responsiveness and eliminates dependency on internet availability.

6.4.8 Mapping and Cloud Synchronization

Advanced systems can integrate cloud-based mapping and fleet management platforms,
allowing:

 Sharing of maps across multiple robots in a hospital.

 Cloud storage of gesture logs, performance data, and usage history.
 Remote monitoring and control by hospital IT staff or medical personnel.

6.4.9 Environmental Adaptability and Multi-Zone Routing

Future robots can be trained to handle different hospital zones with different
protocols:

 Quarantine zones (automated sanitization alerts).

 Operating rooms (limited access).
 Wards (gesture-controlled interaction with patients).

Using mapped routes and geo-fenced regions, the robot can adapt its behavior
accordingly.
57
6.4.10 Fail-Safe Systems and Redundancy

To ensure safety and compliance, future designs should include:

 Dual LiDAR or camera systems for redundancy.

 Emergency stop buttons and override gestures.
 Power failure recovery systems with auto-resume functions.

Battery health and backup monitoring.

6.5 Summary

The development of this gesture-controlled robot for medical assistance reflects the
convergence of embedded systems, computer vision, and healthcare innovation. The
project demonstrates how affordable microcontrollers like ESP32, when combined
with intelligent software frameworks, can lead to smart, accessible, and human-
friendly technologies.

This project not only fulfills its initial goals of offering contactless assistance in
hospital environments but also opens avenues for multidisciplinary advancements.
With further refinement and integration of AI, robotics, and IoT, systems like this
could become an integral part of smart healthcare ecosystems, supporting nurses,
aiding elderly patients, and ensuring safe and hygienic medical service delivery.

58
 REFERENCES

[1] S. S. Raut and R. P. Deshmukh, “Hand Gesture Recognition for Human Computer
Interaction,” International Journal of Science and Research (IJSR), vol. 5, no. 4, pp.
761–764, Apr. 2016.
This paper introduces an image processing-based hand gesture recognition system for
touchless human-computer interaction.

[2] P. Yadav, R. Mahajan, and S. Singh, “Gesture Controlled Robot Using Hand
Gestures and Accelerometer,” International Research Journal of Engineering and
Technology (IRJET), vol. 4, no. 6, pp. 1942–1946, Jun. 2017.
The study presents a robot navigated by hand gestures detected via an accelerometer,
demonstrating wireless control using microcontrollers.

[3] R. Singh and R. Mehta, “Development of Gesture Based Human–Robot Interface

Using OpenCV,” International Journal of Innovative Research in Computer and
Communication Engineering (IJIRCCE), vol. 5, no. 5, May 2017.
The authors developed a vision-based robotic control interface using OpenCV for
real-time gesture interpretation.

[4] A. Gupta, “Medical Robots: Current Applications and Future Scope,” Journal of
Biomedical and Healthcare Engineering, vol. 8, no. 3, 2021.
This review outlines the use of robots in surgery and rehabilitation, with projections
for future advancements in medical robotics.

[5] “Gesture-Controlled Robotic Arm with Mobile Rover Integration,” International

Advanced Research Journal in Science, Engineering and Technology (IARJSET),
2024.
Details a system combining gesture control with mobile rover navigation for remote
manipulation.

[6] “Design and Development of Gesture Controlled Robotic Arm,” International

Journal of Computer Applications Technology and Research (IJCAT), 2023.
Describes the design and implementation of a gesture-controlled robotic arm focusing
on hardware and control logic.

[7] “Real-Time Remote-Controlled Human Manipulation Medical Robot Using IoT,”

Semantic Scholar, 2024.
Introduces an IoT-enabled robot for real-time remote medical interaction and
assistance.

[8] “Design and Implementation of a Wireless Gesture Controlled Robotic Arm,”

International Journal of Research and Analytical Reviews (IJRAR), 2023.
Presents a wireless gesture-controlled robotic arm using embedded systems for
remote actuation.

[9] “Design and Fabrication of Hand Gesture Controlled Wheelchair,” International

Journal of Research Publication and Reviews (IJRPR), 2024.
Focuses on assistive technology for physically challenged individuals through
gesture- based wheelchair control.
59
[10] “Gesture-Controlled Mecanum Wheel Car Using ESP32 and MPU6050,”
Journal of Emerging Technologies and Innovative Research (JETIR), 2024.
Presents a robotic vehicle with Mecanum wheels allowing omnidirectional control via
gesture recognition.

[11] “A Real-Time Gesture Controlled Robot Car,” SpringerLink, 2025. Describes a

robot car system that utilizes real-time gesture inputs for dynamic movement control.

[12] iamvny, “AI-Powered Hand Gesture Controlled Robot,” GitHub Repository,

2024. [Online]. Available: [Link]
Combines MediaPipe, OpenCV, and ESP32 to create an AI-based contactless robotic
control system.

[13] “EchoGlove: Assistive Smart Glove for Gesture-Based Communication and

Health Monitoring,” International Journal for Research in Applied Science and
Engineering Technology (IJRASET), 2025.
A wearable glove that facilitates gesture communication and health monitoring for
speech-impaired users.

[14] “IoT Based Hand Gesture Control Robot,” International Journal of Engineering
Research and Technology (IJERT), 2023.
Discusses IoT-enabled robotic control through hand gestures detected by ESP32 and
MPU6050.

[15] “Review Paper on Gesture Controlled Robot with Robotic Arm,” International
Research Journal of Modernization in Engineering Technology and Science
(IRJMETS), 2023.
Provides a comparative review of gesture-controlled robotic arms and technologies
used.

[16] “Hand Gesture Controlled Robot,” International Journal of Creative Research

Thoughts (IJCRT), 2021.
Explores system integration and design strategies for hand gesture-based robotic
navigation.

[17] “Hand Movement Gesture Wheelchair,” IJCRT, 2023.

Presents a gesture-operated wheelchair for mobility-impaired individuals with focus
on accessibility.

[18] “Android Controlled Smart Wheelchair with Gesture and Voice Control,” World
Academy of Research in Science and Engineering (WARSE), 2020.
Integrates Android, gesture, and voice commands to create a versatile wheelchair
control system.

[19] GazeGrasp, “DNN-Driven Robotic Grasping with Wearable Eye-Gaze

Interface,”
arXiv preprint, 2025.
Fuses DNNs with gaze tracking for intuitive robotic manipulation.

[20] “Design and Analysis of Remote-Controlled Robotic Arms for Medical

Operation Purpose,” IARJSET, 2024.

60
Emphasizes surgical precision in robotic arms designed for remote medical procedures.

[21] “Enhancing the Sense of Attention from an Assistance Mobile Robot by

Improving Eye-Gaze Contact,” IARJSET, 2024.
Explores human-robot interaction improvements via gaze tracking for attention
responsiveness.

[22] “Intelligent Robotic Arm,” IARJSET, 2024.

Presents an adaptive robotic arm that utilizes AI algorithms for intelligent task
performance.

[23] “Design and Simulation of Robot Gripper and Conveyor System for
Workstations,” IARJSET, 2024.
Automates workstation tasks with an integrated robotic gripper and conveyor mechanism.

[24] “Learning to Generate Pointing Gestures in Situated Embodied Conversational

Agents,” IARJSET, 2024.
Focuses on the natural generation of pointing gestures in AI-based conversational
systems.

[25] “Evaluation of Objective Functions for the Optimal Design of an Assistive

Robot,” IARJSET, 2024.
Assesses performance metrics for designing robots that assist disabled users.

[26] “Hand Gesture Recognition with Focus on Leap Motion: An Overview,”

IARJSET, 2024.
Analyzes Leap Motion-based systems for accurate gesture capture and interaction.

[27] N. J. Mitra and B. Chanda, “Vision-Based Real-Time Hand Gesture Recognition

Using Deep Learning,” IEEE Access, vol. 8, 2020.
Proposes a robust deep learning model for real-time hand gesture detection.

[28] M. Asadi-Aghbolaghi et al., “Deep Learning for Hand Gesture Recognition: A

Review,” IEEE Transactions on Human-Machine Systems, vol. 50, no. 9, 2020.
A comprehensive review of deep learning methodologies for gesture recognition and their
challenges.

[29] A. Camurri et al., “Toward Natural Gesture Interaction for Medical Robotics,”
IEEE Robotics and Automation Magazine, vol. 27, no. 2, 2020.
Emphasizes natural interfaces in medical robotics for enhanced human-robot
collaboration.

[30] C. Keskin, “Real-Time Hand Pose Estimation Using Depth Sensors,” Computer
Vision and Image Understanding, Elsevier, 2021.
Describes depth-sensor-based hand pose estimation techniques used in real-time
applications.

[31] A. Mallick and A. Kumar, “OpenCV Python: Real-time hand gesture

recognition,” [Online]. Available: [Link]

61
Online tutorial resource covering OpenCV techniques such as contour detection and
background subtraction.

[32] Espressif Systems, “ESP32 Technical Reference Manual,” [Online]. Available:

[Link]
e_manual_en.pdf
Technical document detailing ESP32 microcontroller architecture, features, and usage.

[33] STMicroelectronics, “L298N Dual H-Bridge Motor Driver Datasheet,” [Online].

Available: [Link]
Datasheet providing specifications for the L298N motor driver module, essential for
motor control systems.

[34] OpenCV Organization, “OpenCV Documentation,” [Online]. Available:

[Link]
Official OpenCV documentation with detailed references on computer vision
functionalities.

[35] Python Software Foundation, “Python Official Documentation,” [Online].

Available: [Link]
Provides comprehensive guidance on Python programming syntax and libraries.

[36] World Health Organization, “WHO Guidelines on Contactless Technologies in

Medical Care,” WHO Reports, 2021.
Outlines the use of contactless technologies such as gesture control in minimizing
infection risk.

[37] Samsung Innovation Campus, “Raspberry Pi and IoT Projects,” Course

Materials. Educational content covering IoT project development using Raspberry Pi
for smart applications.

[38] YouTube Channels, “Gesture Controlled Robot using OpenCV and

Arduino/ESP32,” Techiesms, How To Mechatronics, 2024.
Video tutorials demonstrating step-by-step construction of gesture-controlled robots.

[39] YouTube Tutorials, “Gesture-Controlled Robot Projects Using OpenCV,”

Various Creators, 2024.
A wide array of video tutorials covering hardware, hand tracking, and robot control
mechanisms.

[40] Rohan Nagargoje, “Gesture-Controlled Robot Using ESP32,” GitHub

Repository, 2024. [Online]. Available: [Link]
Includes source code and schematics for a gesture-controlled robot with ESP32.

[41] Nilutpol Kashyap, “ESP32 Gesture Controlled Robot,” GitHub Repository,

2024. [Online]. Available: [Link]
A detailed repository containing code and hardware setup for gesture-controlled ESP32
robots.

[42] iamvny, “AI-Powered Hand Gesture Controlled Robot,” GitHub Repository,

2024. [Online]. Available: [Link]

62
Provides complete documentation for AI-based gesture recognition robot using ESP32
and MediaPipe.

[43] GitHub, “Gesture Controlled Robot Projects,” Available: [Link] A

collection of open-source projects on GitHub demonstrating various implementations
of gesture-controlled robots.

63