“GESTURE CONTROLLED ROBOT”

A
Project Report
Submitted
In partial fulfilment
for the award of the Degree of
Bachelor of technology
in Department of Mechanical Engineering

Project Guide Submitted by:

Mr. Prakash Bhardwaj Aashish Kumar (15ESKME700)
Assistant Prof. Sagar Kumar (15ESKME751)
Rashid Anwar (15ESKME748)
Md Fahad iqubal (15ESKME732)

Department of Mechanical Engineering Swami

Keshvanand Institute of Technology, Management & Gramothan, Jaipur
Rajasthan Technical University
July, 2018

i
 CERTIFICATE

This is to certify that the project report entitled

“GESTURE CONTROLLED ROBOT”
has been submitted by us for fulfillment of the partial requirement of [Link].
Degree in MECHANICAL BRANCH. The work contained in this report is
carried out by us under the guidance of Mr. PRAKASH BHARDWAJ and
under the coordination of Mr. VISHNU JANGID.

Supervisor Name of students

Mr. Vishnu Jangid Aashish Kumar
Assistant Prof. Sagar Kumar
Rashid Anwar
Md fahad iqubal

Project Guide
Mr. Prakash Bhardwaj
Assistant Prof.

ACKNOWLEDGMENT

ii
We express our deep sense of gratitude to our respected and learned guides Mr. Prakash
Bhardwaj and Project Coordinator Mr. Vishnu Jangid for their valuable help and guidance.
We are thankful to them for the encouragement they have given us in completing the project
on gesture-controlled robot.
We are also grateful to respected Dr. N.C Bhandari (HOD) and to our respected Director &
Principal “Jaipal Meel and Ramesh Kumar Pachar” for permitting us to utilize all the
necessary facilities of the institution.

We are also thankful to all other faculty and staff member of our department for their kind
cooperation and help.

Lastly, we would like to express our deep apperception towards our classmate and our in debt
ness to our parents for providing us the moral and encouragement.

Name of students

Aashish Kumar
Sagar Kumar
Rashid Anwar
Md fahad iqubal

iii
 ABSTRACT

A gesture is a form of non-verbal communication in which visible bodily actions

communicate particular messages, either in place of speech or together and in parallel with
words. Gestures include movement of the hands, face, or other parts of the body. Gestures
differ from physical non-verbal communication that does not communicate specific
messages, such as purely expressive displays, proxemics, or displays of joint attention.
Gestures allow individuals to communicate a variety of feelings and thoughts, from contempt
and hostility to approval and affection, often together with body language in addition towards
when they speak. Gesture Controlled Robot is a robot which can be controlled by simple
gestures. The user just needs to wear a gesture device which includes a sensor. The sensor
will record the movement of hand in a specific direction which will result in the movement of
the robot in the respective direction. The robot and the Gesture device are connected
wirelessly via radio waves. The wireless communication enables the user to interact with the
robot in a more friendly way.

1
 CONTENT

Title (i)

Certificate (ii)

Acknowledgment (iii)

Abstract 1

List of figures 4

List of tables 5

CHAPTER 1

1.1 Introduction 6
1.2 robot 7
1.3 human machine interaction 7
1.4 Gesture 8
1.5 motivation for project
8
1.6 objective of project
8
1.7 role of robot 8
1.8 applications
9

CHAPTER 2

2.1 circuit and working 10

2.2 block diagram description 12

2.3 component description

2.3.1 accelerometer (adxl335) 12

2.3.2 rf module 13

2.3.3 encoder (ht12e) 15

2
 2.3.4 decoder ic (ht12d) 16

2.3.5 microcontroller (Arduino Uno) 18

2.4 power 19
2.5 memory 20
2.6 input and output 20
2.6 resistors 21
2.7 comparator ic (lm324) 22
2.8 dc motors 22
2.9 bill of material 23

CHAPTER 3
3.1 working process 24
3.2 interfacing with arduino adxl 335 25
3.3 features 26
3.4 construction and testing 27
3.5 software 27

CHAPTER 4
4.1 input to accelerometer (adxl 335) 28

CHAPTER 5

Literature Review 36

CHAPTER 6

 Conclusion 41
 Limitations 42
 Future Scope 42
 References 43

3
 LIST OF FIGURES

Figure 2.1 gesture-controlled robot 10

Figure 2.2: Accelerometer 12

Figure 2.3-RF Transmitter 13

Figure 2.4-RF Receiver 14

Figure 2-5 encoder (HT12E) 16

Figure 2.6 HT12D IC 17

Figure 2.7 – Arduino 18

Figure2.8 Resister 21

Figure 2.9- LM324 IC 22

Figure 2.10- DC Motor 23

Figure 3.1 interfacing with arduino 25

Figure 4.1 Move forward 28

Figure 4.2 Move right 28

Figure 4.3 Move backward 29

Figure 4.4 Move left 29

Figure 4.5 Input and Output of Comparator IC 30

Fig 4.6 ASK Modulation 31

4
 LIST OF TABLES

Table 1.1- Pin description for RF Tx 14

Table 1.2-Pin description for RF Rx 15

Table 1.3-Pin description for HT12E 16

Table 1.4-Pin description for HT12D 18

Table 1.5-Arduino Specifications 20

Table 4.1-accelerometer orientation 29

Table 4.2 Conditions for this Gesture controlled Robot 30

5
 CHAPTER 1

1.1 INTRODUCTION

Recently, strong efforts have been carried out to develop intelligent and natural interfaces
between users and computer-based systems based on human gestures. Gestures provide an
intuitive interface to both human and computer. Thus, such gesture-based interfaces can not
only substitute the common interface devices, but can also be exploited to extend their
functionality.

Gesture recognition technologies are much younger in the world of today. At this time there
is much active research in the field and little in the way of publicly available
implementations. Several approaches have been developed for sensing gestures and
controlling robots. Glove based technique is a well-known means of recognizing hand
gestures. It utilizes a sensor attached to a glove that directly measures hand movements.

A Gesture Controlled robot is a kind of robot which can be controlled by hand gestures and
not the old-fashioned way by using buttons. The user just needs to wear a small transmitting
device on his hand which includes a sensor which is an accelerometer in our case. Movement
of the hand in a specific direction will transmit a command to the robot which will then move
in a specific direction. The transmitting device includes a Comparator IC for assigning proper
levels to the input voltages from the accelerometer and an Encoder IC which is used to
encode the four-bit data and then it will be transmitted by an RF Transmitter module.

At the receiving end an RF Receiver module will receive the encoded data and decode it by
using a decoder IC. This data is then processed by a microcontroller and passed onto a motor
driver to rotate the motors in a special configuration to make the robot move in the same
direction as that of the hand.

Robots are playing an important role in automation across all the sectors like construction,
military, medical, manufacturing, etc. After making some basic robots like line follower
robot, computer-controlled robot, etc; we have developed this accelerometer-based gesture-
controlled robot by using Arduino Uno. In this project we have used hand motion to drive the
robot. For this purpose, we have used accelerometer which works on acceleration. A gesture-
controlled robot is controlled by using hand in place of any other method like buttons or
joystick. Here one only needs to move hand to control the robot. A transmitting device is

6
used in your hand which contains RF Transmitter and accelerometer. This will transmit
command to robot so that it can do the required task like moving forward, reverse, turning
left, turning right and stop. All these tasks will be performed by using hand gesture.

1.2 ROBOT

A robot is usually an electro-mechanical machine that can perform tasks automatically. Some
robots require some degree of guidance, which may be done using a remote control or with a
computer interface. Robots can be autonomous, semi-autonomous or remotely controlled.
Robots have evolved so much and are capable of mimicking humans that they seem to have a
mind of their own.

1.3 HUMAN MACHINE INTERACTION

An important aspect of a successful robotic system is the Human-Machine interaction. In the

early years the only way to communicate with a robot was to program which required
extensive hard work. With the development in science and robotics, gesture-based
recognition came into life. Gestures originate from any bodily motion or state but commonly
originate from the face or hand. Gesture recognition can be considered as a way for computer
to understand human body language. This has minimized the need for text interfaces and
GUIs (Graphical User Interface) Gesture controlled robot moves according to hand
movement as we place transmitter in our hand. When we tilt hand in front side, robot start to
moving forward and continues moving forward until next command is given.

 When we tilt hand in backward side, robot change its state and start moving in
backwards direction until other command is given.
 When we tilt it in left side Robot get turn left till next command.
 When we tilt hand in right side robot turned to right.
 And for stopping robot we keep hand in stable.

1.4 GESTURE

A gesture is an action that has to be seen by someone else and has to convey some piece of
information. Gesture is usually considered as a movement of part of the body, esp. a hand or
the head, to express an idea or meaning.

7
1.5 MOTIVATION FOR PROJECT

Our motivation to work on this project came from a disabled person who was driving his
wheel chair by hand with quite a lot of difficulty. So we wanted to make a device which
would help such people drive their chairs without even having the need to touch the wheels of
their chairs.

1.6 OBJECTIVE OF PROJECT

Our objective is to make this device simple as well as cheap so that it could be mass produced
and can be used for a number of purposes.

1.7 ROLE OF ROBOT

Robots are playing an important role in automation across all the sectors like construction,
military, medical, manufacturing, etc. After making some basic robots like line follower
robot, computer-controlled robot, etc; we have developed this accelerometer-based gesture-
controlled robot by using Arduino Uno. In this project we have used hand motion to drive the
robot. For this purpose, we have used accelerometer which works on acceleration. A gesture-
controlled robot is controlled by using hand in place of any other method like buttons or
joystick. Here one only needs to move hand to control the robot. A transmitting device is
used in your hand which contains RF Transmitter and accelerometer. This will transmit
command to robot so that it can do the required task like moving forward, reverse, turning
left, turning right and stop. All these tasks will be performed by using hand gesture.

Here the most important component is accelerometer. Accelerometer is a 3-axis acceleration

measurement device with ±3g range. This device is made by using polysilicon surface sensor
and signal conditioning circuit to measure acceleration. The output of this device is Analog in
nature and proportional to the acceleration. This device measures the static acceleration of
gravity when we tilt it and gives a result in form of motion or vibration.

8
1.8 APPLICATIONS

 Through the use of gesture recognition, remote control with the wave of a hand of
various devices is possible.
 Gesture controlling is very helpful for handicapped and physically disabled people to
achieve certain tasks, such as driving a vehicle.
 Gestures can be used to control interactions for entertainment purposes such as
gaming to make the game player's experience more interactive or immersive.
 Traditional interfaces, keyboards and mice present a bottleneck in application that rely
on heavy interaction of the user with the machine due to the unnaturalness of the
interaction.
 From reading lots of related articles, we have learnt that recent efforts have attempted
to eliminate this bottleneck by developing different ways of interacting with
computers, for example: speech, handwriting.

9
 CHAPTER 2

2.1 Circuit and working

Our gesture-controlled robot works on the principle of accelerometer which records hand
movements and sends that data to the comparator which assigns proper voltage levels to the
recorded movements. That information is then transferred to a encoder which makes it ready
for RF transmission. On the receiving end, the information is received wirelessly via RF,
decoded and then passed onto the microcontroller which takes various decisions based on the
received information. These decisions are passed to the motor driver ic which triggers the
motors in different configurations to make the robot move in a specific direction. The
following block diagram helps to understand the working of the robot:

Fig 2.1 gesture-controlled robot block diagram[1]

shows the block diagram of the complete project. The movement of gesture transmitter
section is sensed by the accelerometer and the voltage levels corresponding to movement of
the robot along X and Y axes are compared by a comparator, following which corresponding
instructions are transmitted. through the RF transmitter to control the robot. The robot

10
receives the instructions. The microcontroller drives motors corresponding to the instructions
received and displays the related message on the LCD. the circuit of the gesture transmitter.
It is built around encoder HT12E (ICl), precision voltage comparator LM339 (IC2), regulator
7805 (IC3), RF transmitter module, accelerometer module and a few discrete components
Pins 9 and 10 of comparator IC2 are connected to X-axis output pin of the accelerometer.
Voltage levels corresponding to the motion along X-axis are continuously compared with a
reference voltage set oy presets VR3 and VR4 using op-amps A3 and A4. Similarly, for Y-
axis, voltage levels corresponding to the motion along Y axis are continuously compared with
a reference voltage set by presets VRI and VR2 using op-amps Al and A2. If the voltage level
for any axis crosses the predefined voltage level, the corresponding op-amp's output goes
low, generating a different four-bit code from the output of all four op-amps of IC2 for
motion along each axis. This four-bit code is fed to pins Ad 8 through AD ll of encoder ICI.
The code is also indicated by LEDI through LED4, respectively. Encoder lCI encodes the
code, which is then transmitted using RF transmitter module TXI. Regulator 7805 provides
regulated 5V supply to the circuit. Accelerometer module. An accelerometer is an electro
mechanical device that can measure acceleration of anything that it is mounted on. The
accelerometer module used here is based on ADXL335 triple-axis accelerometer from
Analog Devices. Robot. shows the circuit of the wireless robot. It comprises RF receiver
module RXI, regulator 7805 (lC4), decoder HT12D (lC5), microcontroller AT89C52 (lC6),
motor driver L293D (IC7) and a few discrete components. Microcontroller AT89C52 is the
heart of the circuit. It is a low-power, high-performance CMOS eight-bit microcontroller that
has 8 kB of Flash, 256 bytes of RAM, 32 input/ output (I/O) lines, three l6-bit timers/
counters, a six-vector two-level interrupt architecture, a full.

We divided our task into two parts to make the task easy and simple and to avoid complexity
and make it error free. The first is the transmitting section which includes the following
components:

 Accelerometer
 Comparator IC
 Encoder IC
 RF Transmitter Module

The second is the receiving end which comprises of following main components:

 RF Receiver Module

11
  Decoder IC
 Arduino
 Motor Driver IC
 DC Geared Motors

2.2 BLOCK DIAGRAM DESCRIPTION

 Accelerometer placed on the hand sensed the tilt made by the hand.
 Accelerometer capable of measuring how fast the speed of object is changing.
 This tilt corresponded to the analog voltage.
 Using this voltage, control signals are generated for four directions of the robot car.

2.3 COMPONENT DESCRIPTION

2.3.1 Accelerometer (ADXL335)

Fig.2.2: Accelerometer[1]

An Accelerometer is an electromechanical device that measures acceleration forces. These

forces may be static, like the constant force of gravity pulling at your feet, or they could be
dynamic – caused by moving or vibrating the accelerometer. It is a kind of sensor which
record acceleration and gives an analog data while moving in X, Y, Z direction or may be X,
Y direction only depending on the type of the sensor.

2.3.2 RF Module

RF stands for Radio Frequency. This module consists of further two parts: Transmitter (Tx)
and Receiver (Rx). It is a small electronic device used to transmit and receive signal between
two devices. It is available in different operating frequencies with different operating range.
An Encoder Circuit & a Decoder Circuit is used along with the Transmitter & Receiver
respectively in order to transmit & receive the message/signal.

12
Radio frequency (RF) is a rate of oscillation in the range of about 3 KHz to 300 GHz, which
corresponds to the frequency of radio waves, and the alternating currents which carry radio
signals.

Although radio frequency is a rate of oscillation, the term "radio frequency" or its
abbreviation "RF" are also used as a synonym for radio – i.e. to describe the use of wireless
communication, as opposed to communication via electric wires. The RF module is working
on the frequency of 434 MHz and has a range of 50-80 meters.

Figure 2.3-RF Transmitter[1]

Table 1.1- Pin description for RF Tx

13
 Figure 2.4-RF Receiver[1]

Table1.2-Pin description for RF Rx

2.3.3 ENCODER (HT12E)

HT12E is an encoder integrated circuit of 212 series of encoders. They are paired with 212
series of decoders for use in remote control system applications. It is mainly used in
interfacing RF and infrared circuits. The chosen pair of encoder/decoder should have same
number of addresses and data format. Simply put, HT12E converts the parallel inputs into

14
serial output. It encodes the 12-bit parallel data into serial for transmission through an RF
transmitter. These 12 bits are divided into 8 address bits and 4 data bits.

HT12E has a transmission enable pin which is active low. When a trigger signal is received
on TE pin, the programmed addresses/data are transmitted together with the header bits via
an RF or an infrared transmission medium. HT12E begins a 4-word transmission cycle upon
receipt of a transmission enable. This cycle is repeated as long as TE is kept low. As soon as
TE returns to high, the encoder output completes its final cycle and then stops.

Figure 2-5 ENCODER (HT12 E) [1]

Table 1.3-Pin description for HT12E

15
2.3.4 DECODER IC (HT12D)
HT12D is a decoder integrated circuit that belongs to 212 series of decoders. This series of
decoders are mainly used for remote control system applications, like burglar alarm, car door
controller, security system etc. It is mainly provided to interface RF and infrared circuits.
They are paired with 212 series of encoders. The chosen pair of encoder/decoder should have
same number of addresses and data format.

In simple terms, HT12D converts the serial input into parallel outputs. It decodes the serial
addresses and data received by, say, an RF receiver, into parallel data and sends them to
output data pins. The serial input data is compared with the local addresses three times
continuously. The input data code is decoded when no error or unmatched codes are found. A
valid transmission in indicated by a high signal at VT pin.

HT12D is capable of decoding 12 bits, of which 8 are address bits and 4 are data bits. The
data on 4 bit latch type output pins remain unchanged until new is received.

Figure 2.6- decoder IC [1]

16
 Table 1.4-Pin description for HT12D [2]

2.3.5 MICROCONTROLLER (ARDUINO UNO)

Arduino Uno is a microcontroller board based on the ATmega328P. It has 14 digital

input/output pins (of which 6 can be used as PWM outputs), 6 analog inputs, a 16 MHz
quartz crystal, a USB connection, a power jack, an ICSP header and a reset button. It contains
everything needed to support the microcontroller; simply connect it to a computer with a
USB cable or power it with an AC-to-DC adapter or battery to get started.

"Uno" means one in Italian and was chosen to mark the release of Arduino Software (IDE)
1.0. The Uno board and version 1.0 of Arduino Software (IDE) were the reference versions of
Arduino, now evolved to newer releases. The Uno board is the first in a series of USB
Arduino boards, and the reference model for the Arduino platform; for an extensive list of
current, past or outdated boards see the Arduino index of boards.

Figure 2.7 – Arduino [2]

17
2.4 Power
 The Arduino Uno board can be powered via the USB connection or with an external
power supply. The power source is selected automatically.
 External (non-USB) power can come either from an AC-to-DC adapter (wallwart) or
battery. The adapter can be connected by plugging a 2.1mm contrapositive plug into
the board's power jack. Leads from a battery can be inserted in the GND and Vin pin
headers of the POWER connector.
 The board can operate on an external supply from 6 to 20 volts. If supplied with less
than 7V, however, the 5V pin may supply less than five volts and the board may
become unstable. If using more than 12V, the voltage regulator may overheat and
damage the board. The recommended range is 7 to 12 volts.

The power pins are as follows:

Vin: The input voltage to the Arduino/Genuino board when it's using an external power
source (as opposed to 5 volts from the USB connection or other regulated power source). You
can supply voltage through this pin, or, if supplying voltage via the power jack, access it
through this pin.

VT: This pin outputs a regulated 5V from the regulator on the board. The board can be
supplied with power either from the DC power jack (7 - 12V), the USB connector (5V), or
the VIN pin of the board (7-12V). Supplying voltage via the 5V or 3.3V pins bypasses the
regulator, and can damage your board. We don't advise it. 3V3. A 3.3-volt supply generated
by the on-board regulator. Maximum current draw is 50 mA.

GND. Ground pins.

IOREF. This pin on the Arduino/Genuino board provides the voltage reference with which
the microcontroller operates. A properly configured shield can read the IOREF pin voltage
and select the appropriate power source or enable voltage translators on the outputs to work
with the 5V or 3.3V.

18
 Table 1.5-Arduino Specifications [2]

Microcontroller ATmega328P
Operating Voltage 5V
Input Voltage (recommended 7-12V
Input Voltage (limit) 6-20V
Digital I/O Pins 14 (of which 6 provide PWM output)
PWM Digital I/O Pins 6
Analog Input Pins 6
DC Current per I/O Pin 20 mA
DC Current for 3.3V Pin 50 mA
Flash Memory 32 KB (ATmega328P) of which 0.5 KB
used by bootloader
SRAM 2 KB (ATmega328P)
EEPROM 1 KB (ATmega328P
Clock Speed 16 MHz
LED_BUILTIN 13
Length 68.6 mm
Width 53.4 mm

2.5 Memory

The ATmega328 has 32 KB (with 0.5 KB occupied by the bootloader). It also has 2 KB of
SRAM and 1 KB of EEPROM (which can be read and written with the EEPROM library).

2.6 Input and Output

See the mapping between Arduino pins and ATmega328P ports. The mapping for the
Atmega8, 168, and 328 is identical.

2.6 Resistors:

19
This is the most common component in electronics. It is used mainly to control current and
voltage within the circuit. You can identify a simple resistor by its simple cigar shape with a
wire lead coming out of each end. It uses a system of colour coded bands to identify the value
of the component (measured in Ohms) *A surface mount resistor is in fact mere millimetres
in size but performs the same function as its bigger brother, the simple Resistor. A
potentiometer is a variable resistor. It lets you vary the resistance with a dial or sliding
control in order to alter current or voltage on the fly. This is opposed to the “fixed” simple
resistors.

Resistor values - the resistor colour code. Resistance is measured in ohms, the symbol for
ohm is an omega .1 is quite small so resistor values are often given in k and M .
1k = 1000 1M = 1000000 .Resistor values are normally shown using coloured
bands.
Each colour represents a number as shown in the table.

Most resistors have 4 bands:

 The first band gives the first digit.

 The second band gives the second digit.
 The third band indicates the number of zeros.
 The fourth band is used to shows the tolerance (precision) of the resistor, this may be
ignored for almost all circuits but further details are given below.

Figure 2.8 Resister [3]

Resistor values - the resistor colour code

20
2.7 COMPARATOR IC (LM324)
The comparator ic compares the analog voltage received from the accelerometer and
compares it with a reference voltage and gives a particular high or low voltage. The received
signal is quite noisy and of various voltage levels. This ic compares those levels and outputs
in the form of 1 or 0 voltage level. This process is called signal conditioning.

The figure shown below is comparator IC. The pins 1, 7, 8 and 14 are output pins. A
reference voltage is connected to the negative terminal for high output when input is high or
positive terminal for high output when input is low from the LM324 IC.

Figure 2.9- LM324 IC [3]

2.8 DC MOTORS

21
A machine that converts DC power into mechanical power is known as a DC motor. Its
operation is based on the principle that when a current carrying conductor is placed in a
magnetic field, the conductor experiences a mechanical force.

DC motors have a revolving armature winding but non-revolving armature magnetic field
and a stationary field winding or permanent magnet. Different connections of the field and
armature winding provide different speed/torque regulation features. The speed of a DC
motor can be controlled by changing the voltage applied to the armature or by changing the
field current.

Figure 2.10- DC Motor [3]

2.9 BILL OF MATERIAL

22
[Link] COMPONENT SPECIFICATION QUANTITY PRICE
1 PLYWOOD 0.6m*0.45m 1 150
2 CARDBOARD 0.15m*0.13m 1 50
3 BATTERY 9V, 4V 3 200
4 MICROCONTROLLER AT89C52 1 1225
5 ENCODER HT-12E 1 130
6 DECODER HT12D 1 160
7 MOTER DRIVE L293D 2 94
7 WHEEL 0.6m 2 120
8 CASTOR WHEEL 360 degree 1 180
9 MOTOR 60 rpm 2 500
10 CAPACITOR 100mf, 0.1 mf 5 100
11 RESISTOR 1000 ohm 12 84
12 VOLTAGE Input= 35V 2 48
REGULATOR Output=5V
13 LED Colour 6 30
14 CRYSTAL 32.768KHz 1 20
OSCILLATOR
15 RF TRANSMITTOR 433.92MHz 1 150
16 RF RECIVER 433.92 MHz 1 200
17 POTENTIOMETER 100k 4 140
19 WIRE 5m 1 50
20 COMPAROTOR LM399 1 150
TOTAL 4071

CHAPTER 3

23
3.1 Working Process
In this project, a simple demonstration of RF Communication with the help of Arduino UNO
boards is given. The aim of the project is to successfully transmit data between the RF
Transmitter – Receiver modules using two Arduino UNO microcontroller boards. The
working of the project is explained here.

 Note: The project can be implemented with or without the help of a special library
called “VirtualWire.h”. The project implemented here uses the library. If we want to
implement the project without the library, then we need to change the receiver part of
the circuit.
 VirtualWire.h is a special library for Arduino created by Mike McCauley. It is a
communication library that allows two Arduino’s to communicate with each other
using RF Module i.e. transmitter – receiver pair. This library consists of several
functions that are used for configuring the modules, transmission of data by the
transmitter module and data reception by the receiver module.
 In this project, the transmitter simply sends two characters i.e. it sends the character
“1” and with a delay of few seconds, it sends the character “0”. Whenever the “1” is
sent, the LED on the transmitting side of the project will be turned ON. As this “1” is
transmitted via RF communication, the receiver will receive the data “1”.
 When the receiver receives “1”, the Arduino on the receiver side of the project will
turn ON the LED on its side.
 Similarly, when the data “0” is transmitted by the RF transmitter, the LED on the
transmitter side is turned OFF. As a result, the receiver now receives “0” and the LED
on the receiver side is also turned OFF.
 Hence, the receiver is imitating the actions of the transmitter.

3.2 interfacing with arduino adxl 335

24
 Figure 3.1 interfacing with Arduino

The accelerometer module has 5 pins, namely

 GND-To be connected to Arduino's GND
 VCC-To be connected to Arduino's 5V

25
  X-To be connected to Analog Pin A5
 Y-To be connected to Analog Pin A4
 Z-To be connected to Analog Pin A3

NOTE: We don't need to power the module from 3.3v because it already has a 5v to 3.3v
converter. Use 2-pin relegate for connecting Vcc and GND. Use a 3-pin relegate for
connecting X, Y & Z outputs. Also connect AREF pin to the 3.3v. This is done to set the
reference voltage to 3.3v because the output of ADXL335 is 3.3v compatible.

3.3 FEATURES
Traditional interfaces, keyboards and mice present a bottleneck in application that rely on
heavy interaction of the user with the machine due to the unnaturalness of the interaction.

 From reading lots of related articles, we have learnt that recent efforts have attempted
to eliminate this bottleneck by developing different ways of interacting with
computers, for example: speech, handwriting.
 Through the use of gesture recognition, remote control with the wave of a hand of
various devices is possible.
 Gesture controlling is very helpful for handicapped and physically disabled people to
achieve certain tasks, such as driving a vehicle.
 Gestures can be used to control interactions for entertainment purposes such as
gaming to make the game player's experience more interactive or immersive

3.4 Construction and testing

Actual-size, single-side PCBs for the gesture transmitter and the wireless robot are shown in
Figs 5 and 7 and their component layouts in Figs 6 and 8, respectively. Assemble the circuits
on PCBs as this saves time and minimises assembly errors. Carefully assemble the
components and double check for any overlooked error. Use IC bases for the ICs. Before
inserting the microcontroller and other ICs in their bases, check the correct supply voltage.
Suitable connectors are provided on the PCB to connect geared motors M1 and M2. Connect
the motor and power the circuit with a 12V battery. Using presets VRI through VR4, adjust
its sensitivity as per your liking. You can now make hand gestures as shown in Fig. 1 to move
the robot in all directions. To test the circuit, check voltages as per the test-point table.

3.5 Software

26
The source program is written in 'C' language and compiled using Keil J. lVision4 compiler
to generate the Intel hex code. The generated hex code is burnt into the microcontroller using
a suitable programmer. It is well commented and easy to understand.

CHAPTER 4

IMPLEMENTATION

4.1 INPUT TO ACCELEROMETER (ADXL 335)

27
Different Hand gestures to make the robot move in specific directions are as follow:

Figure 4.1 Move forward

Figure 4.2 Move right

28
 Figure 4.3 Move backward

Figure 4.4 Move left

The robot only moves when the accelerometer is moved in a specific direction. The valid
movements are as follows:

Table 4.1-Accelerometer orientation[3]

The accelerometer records the hand movements in the X and Y directions only and outputs
constant analog voltage levels. These voltages are fed to the comparator IC which compares
it with the references voltages that we have set via variable resistors attached to the IC. The
levels that we have set are 1.7V and 1.4V. Every voltage generated by the accelerometer is
compared with these and an analog 1 or 0 signal is given out by the comparator IC.

29
 Figure 4.5 Input and Output of Comparator IC[4]

There are total five conditions for this Gesture controlled Robot which are giving below:

Table 4.2 Conditions for this Gesture controlled Robot

This analog signal is the input to the encoder IC. The input to the encoder is parallel while the
output is a serial coded waveform which is suitable for RF transmission. A push button is
attached to pin 14 of this IC which is the Transmission Enable (TE) pin. The coded data will
be passed onto the RF module only when the button is pressed. This button makes sure no
data is transmitted unless we want to.

30
 The RF transmitter modulates the input signal using Amplitude Shift Keying (ASK)
modulation. It is the form of modulation that represents digital data as variations in the
amplitude of a carrier wave.
ret=Rxmsg();
The following figure shows the }
modulated output of the RF module: }

//----------------------------------------------------

// Retry subroutine to check the

connectivity of modem
Fig 4.6 modulation
//----------------------------------------------------
The RF modules works on the void Retry(void)
frequency of 315MHz. It means that the {
carrier frequency of the RF module is unsigned char i=0;
315MHz. The RF module enables the retry:

user to control the robot wirelessly and lcdcmd(0x01);

with ease. Delay Ms (10);

lcdcmd(0x80);

DelayMs(10);

for(I=0 ;i<5;i++)

b[i]=0x00;

i=0;

while(a[i]! ='\0')
programme {

SBUF=a[i];
#include<reg52.h> while (TI==0); // AT sending
sbit RS=P3^7; TI=0;
sbit EN=P3^6; lcdData(a[i]);
//--------------------------------------- i++;
// Forward function declaration }
//--------------------------------------- lcdData (' '); // Enter
void Retry(void); SBUF=0x0d;
void Txmsg(unsigned char k); while (TI==0);
unsigned char Rxmsg(void); TI=0;
void lcdinit(void); For (i=0;i<5;i++) //command to race data
void lcdData(unsigned char l); {

31
void lcdcmd(unsigned char k);

unsigned char code welcome1[16] ="Welcome

to all"; i=0;

unsigned char code welcome2[16] while d[i]=0x00; ="In our

project";
(a [i]! ='\0')
//---------------------------------------
{
// Main rotine
SBUF=a[i];
//---------------------------------------
while (TI==0); // ATE0
void main () sending to turn off the
echo
{
TI=0;
unsigned char ret;
lcdData(a[i]);
TMOD=0x20; // Configure UART at 9600 baud
rates i++;

TH1=0xFD; }

SCON=0x50; lcdData (' '); // Enter

TR1=1; SBUF=0x0d;

Lcdinit (); // Initialize LCD while (TI==0);

antiplay (); TI=0;

DelayMs (5000); }

Intimate (); // Initialize Modem Infits:

while(1) for(i=0; i<6;i++)

{ d[i]=0x00;

{ lcdcmd(0x01);

if(j>=1000) DelayMs (10);

lcdcmd(0x80);
goto retry;
DelayMs (10);
DelayMs (1);
i=0;
j++;
while(b[i]! ='\0')
}
{
b[i]=SBUF; SBUF=b[i];
RI=0; while (TI==0); // sending
AT+CPMS to set the
lcdData(b[i]); preffered memory location
} TI=0;

DelayMs (100); lcdData(b[i]);

i++;

32 }

lcdData(' '); // Enter

for (i=0; i<5; i++) //command to
recv data return ret;
if(j>=1000)

{ }goto ReInit;

for (i=0; i<102;i++)

DelayMs(1);
if(b[i]=='E' || b[i]=='R')
{j++;
goto retry;
got
} success;
if(b[i]=='O' || b[i]=='K')
}d[i]=SBUF;
{
got retry;
RI=0;
return; i=i+8;
lcdData(d[i]);
} while(c[i+4]!
} ='O' && c[i+5]!
='K')
} DelayMs(100);
{
goto retry; }
lcdData(c[i]);
} return;
i++;
}
//--------------------------------------- DelayMs (100);
//---------------------------------------
}
// Recieve message subroutine
//--------------------------------------- DelayMs (10000);
//---------------------------------------
void InitModem(void) delete:
unsigned char Rxmsg(void)
for (i=0; i<102; i++)
{ {
{
unsigned int j=0; lcdData('t');
c[i]=0x00;
unsigned char i=0,k=0; SBUF='t';
}
unsigned char d[6]; while(TI==0);
SBUF='a';
ReInit: TI=0;
62
lcdData('m');
//Retry (); while (TI==0);
SBUF='m';
57 TI=0;
while(TI==0);
For (i=0; i<6;i++) lcdData('t');
TI=0;
while (TI==0); SBUF='t';
lcdData('g');
while (TI==0);
TI=0; 60
TI=0;
for (i=0; i<5; i++) //command SBUF='g'; to
recv data lcdData ('+');
while(TI==0);
SBUF='+';
{ TI=0;
while (TI==0);
j=0; lcdData('r');
TI=0;
while(RI==0) SBUF='r';
lcdData('c');

SBUF='c';
33
 //-------------------------------
{ --------

void lcdcmd (unsigned

if(j>=1000)
char k)
got Rein it; {
DelayMs (1); P1=k;

j++; DelayMs (1);

} EN=0;

return;
d[i]=SBUF;
}
RI=0;
//------------------------------------
lcdData(d[i]); ---

} // Delay mS function

for (i=0; i<5; i++) //command //------------------------------------ to compare

---
data
void DelayMs (unsigned int
{ count)
if((d[i]=='E') || (d[i]=='R')) {// mSec Delay

got Rein it; 11.0592 MHz

if((d[i]=='+') || (d[i]=='C')) unsigned int i; // Keil v7.5a

while(count) {
got InitS2;
i = 115; // 115 exact value
}
while(i>0) i--;
InitS2:
count--;
for (k=0; k<10; k++)
}
{ }
for (i=0; i<6; i++) void antiplay (void)

d[i]=0x00; {

} unsigned char d=0, a

[]=".";
for (i=0; i<5;i++) //command to recv
while (d! =15)
data
{
{
lcdData(a[d]);
j=0;
d++;
while(RI==0) }

lcdcmd(0xc0);

while (TI==0);

34
TI=0;
lcdData('g');
SBUF='g';
while (TI==0);
TI=0;
lcdData('d');
SBUF='d';
while (TI==0);
TI=0;
lcdData ('=');
SBUF='=';
while (TI==0);
TI=0;
lcdData ('1');
SBUF='1';
while (TI==0);
TI=0;
lcdData(0x0d); // Enter
SBUF=0x0d;
while (TI==0);
TI=0;
for(i=0; i<5;i++) //command to rich data
{
j=0;
while (RI==0)
{
if(j>=1000)
got delete;
DelayMs (1);
j++;

35
}
c[i]=SBUF;
RI=0;
lcdData(c[i]);
}
DelayMs (1000);
return ret;
}
//---------------------------------------
// Lcd initialization subroutine
//---------------------------------------
void
}
//---------------------------------------
// Lcd data display
//---------------------------------------
void lcdData (unsigned char l)
{
EN=0;
return;
}
//---------------------------------------
// Lcd command

CHAPTER 5
Literature Review

1. Moniruzzaman Bhuiyan and Rich Picking in Centre for Applied Internet Research
(CAIR), Glyndŵr University, Wrexham, UK proposed a review of the history of Gesture
controlled user interface (GCUI), and identifies trends in technology, application and
usability. Their findings conclude that GCUI affords realistic opportunities for specific
application areas, and especially for users who are uncomfortable with more commonly used

36
input devices. They have tried collated chronographic research information which covers the
past 30 years. They investigated different types of gestures, its users, applications,
technology, issues addressed, results and interfaces from existing research. They consider the
next direction of gesture-controlled user interfaces as rich user interface using gestures seems
appropriate for current and future ubiquitous and ambient devices.

Moniruzzaman Bhuiyan, Rich Picking of Institute of Information technology, University of

Dhaka, Dhaka, Bangladesh; Centre for Applied Internet Research, Glyndwr University,
Wrexham, United Kingdom on September 2011 in Journal of Software Engineering and to
meet the challenges of ubiquitous computing, ambient technologies and an increasingly older
population, researchers have been trying to break away from traditional modes of interaction.
A history of studies over the past 30 years reported in this paper suggests that Gesture
Controlled User Interfaces (GCUI) now provide realistic and affordable opportunities, which
may be appropriate for older and disabled people. They have developed a GCUI prototype
application, called Open Gesture, to help users carry out everyday activities such as making
phone calls, controlling their television and performing mathematical calculations. Open
Gesture uses simple hand gestures to perform a diverse range of tasks via a television
interface. They describe Open Gesture and reports its usability evaluation. They conclude that
this inclusive technology offers some potential to improve the independence and quality of
life of older and disabled users along with general users, although there remain significant
challenges to be overcome.

2. Stefan Waldherr, Roseli Romero, Sebastian Thrun describes a gesture interface for the
control of a mobile robot equipped with a manipulator. The interface uses a camera to track a
person and recognize gestures involving arm motion. A fast, adaptive tracking algorithm
enables the robot to track and follow a person reliably through office environments with
changing lighting Prof Kamal K Vyas Director (SIET) SGI Sikar kamalkvyas@[Link]
Amita Pareek Tech-Advisor, eYuG amushiva1@[Link] Dr Sandhya Vyas HOD (S & H),
BBVP Pilani profsandhyavyas@[Link] International Journal on Recent and Innovation
Trends in Computing and Communication Volume: 1 Is conditions. Two alternative methods
for gesture recognition are compared: a template-based approach and a neural network
approach. Both are combined with the Viterbi algorithm for the recognition of gestures
defined through arm motion (in addition to static arm poses). Results are reported in the
context of an interactive clean-up task, where a person guides the robot to specific locations
that need to be cleaned and instructs the robot to pick up trash.

37
3. J. Koster of University of Twente, Human Media Interaction, March 2006 proposed
that gestures are not often used to control domestic appliances in a modern household. They
describes a research project on the current use of gestures in domestic appliances and possible
usage in the future. First, a literature survey is done in different gesture input possibilities,
also advantages and disadvantages are discussed. After that, a few current domestic
appliances with their input possibilities are reviewed. To give an overview in the applied
gesture field, a research in domestic appliances with the use of gestures is presented. Using
the described input methods, new gesture control methods are introduced in the domestic
field. A remote control with touchpad or capacitive field controls came out as an interesting
device. A small qualitative user study is done with a concept 2D paper model of a remote
control to evaluate its usability. Seven users from different ages executed tasks and a small
questionnaire to get their opinions. It is concluded that a remote control with a touch sensitive
interface in combination with a displayed remote-control screen is preferred above the
traditional remote-control using push buttons.

4 Rafiqul Zaman Khan and Noor Adnan Ibraheem of Department of Computer Science,
A.M.U. Aligarh, India in International Journal of Artificial Intelligence & Applications
(IJAIA), in July 2012 proposed hand gesture recognition: a literature review. They told that
hand gesture recognition system received great attention in the recent few years because of its
manifoldness applications and the ability to interact with machine efficiently through human
computer interaction. They presented a survey of recent hand gesture recognition systems.
Key issues of hand gesture recognition system are presented with challenges of gesture
system.

Xin Wei Sha, Giovanni Iachello, Steven Dow, Yoichiro Serita, Tazama St. Julien, Julien
Fistre of School of Literature, Communication, Computing and Culture / GVU Center
Georgia Institute of Technology proposed Continuous Sensing of Gesture for Control of
Audio-Visual Media. They describe how continuous sensing of gesture can be achieved by
using low power wireless sensors to enable expressive control of real-time generation of
audio and visual media. Requirements for this type of application are analysed, and our
customized Berkeley Tiny OS motes are compared with other existing wireless sensing
platforms. The software and hardware architecture is discussed, together with an analysis of
its features and limitations and the development challenges they faced. Finally, hints are
given to what kind of sensor networks gesture controlled AV media systems will need in the
near future.

38
5 Rajesh Kannan Megalingam, Sai Manoj Prakhya, Ramesh Nammily Nair, Mithun
Mohan of Amrita Vishwa Vidyapeetham, Amritapuri, Clappana, Kerala, India proposed
Unconventional Indoor Navigation: Gesture Based Wheelchair Control. They told that, there
are large numbers of people in the world with debilitating physical disabilities and elderly
who experience significant difficulties in performing even very basic tasks such as
locomotion, speaking, writing etc. The worst affected class of physically challenged are those
who have become paralyzed over a significant percentage of their bodies, i.e. quadriplegics.
These people find it extremely difficult to perform any task that requires even small amount
of force. So they proposed a gesture based wheelchair for the benefit of physically challenged
and elderly to navigate inside house without much effort. Our proposed system uses a small
camera mounted very close to the user‟s hand, which tracks the small movements of their
fingers to understand the direction of movement of the wheelchair. A recognition system
which identifies the gesture is then interfaced to the wheelchair control system in order to
move it to the desired location. For point to point navigation without human intervention they
propose two unconventional methods: Line Following Navigation (LFN) and Location Aware
and Remembering Navigation (LARN). These two methods are simple and cost effective for
indoor positioning or navigation.

[Link] R. New Knowledge Systems Laboratory Jacksonville State University

Jacksonville proposed a method of temporal hand gesture recognition Ongoing efforts at his
laboratory have been aimed at developing techniques that reduce the complexity of
interaction between humans and computers. In particular, he have investigated the use of a
gesture recognition system to support natural user interaction while providing rich
information content. He present additions to a platform independent, gesture recognition
system which previously tracked hand movement, defined orientation, and determined the
number of fingers being held up in order to control an underlying application. The additions
provide the functionality to determine a temporal gesture, such as movement of the hand in a
circle or square. This technique has proven to replace the common “clickfest” of many
applications with a more powerful, context-sensitive, and natural interface. In addition, the
Simplified Fuzzy ARTMAP (SFAM) learning system could easily be trained to recognize
new gestures that the user creates and map these to a specific action within the application.
He also present sample images and results.

[Link]. R. V. Dharaskar S. A. Chhabria Sandeep Ganorkar proposed robotic arm control

using gesture and voice in International Journal of Computer, Information Technology &

39
Bioinformatics (IJCITB) They told that human-robot voice interface play a key role in many
application fields. Hand gesture is a very natural form of human interaction and can be used
effectively in human computer interaction (HCI). They propose a “Human Machine
Interfacing Device” utilizing hand gestures to communicate with computers and other
embedded systems acting as an intermediary to an appliance. Developments in field of
communication have enabled computer commands being International Journal on Recent and
Innovation Trends in Computing and Communication executed using hand gestures. They
discusses hand glove-based techniques that use sensors to measure the positions of the fingers
and the position of the hand in real-time. Interaction using gesture technology for effective
communication empowering physically challenged to interact with machines and computing
devices including 3-D graphic interactions and simulations. They focus on wireless data
gloves that are proposed to be used for gesture recognition and accordingly robot movement
will take place.

[Link] R. McMillan Air Force Research Laboratory (AFRL/HECP) proposed the

technology and application of gesture based control. He told the technology for using hand,
body and facial gestures as a means for interacting with computers and other physical
devices. He discusses the rationale for gesture based control technology, methods for
acquiring and processing such signals from human operators, applications of these control
technologies, and anticipated future developments. Junhaeng Lee, T. Delbruck, Paul K. J.
Park, Michael Pfeiffer, Chang-Woo Shin1, Hyunsurk Ryu, and Byung Chang Kang Samsung
Advanced Inst. of Technology, Samsung Electronics Co. Ltd. Republic of Korea Institute of
Neuro informatics, University of Zurich and ETH Zurich, Switzerland proposed gesture-
based remote control using stereo pair of dynamic vision sensors. They describe a novel
gesture interface based on a stereo pair of event-based vision sensors and neuro morphic
event processing techniques. The motion trajectory of a moving hand is detected every 3 ms
by spatiotemporally correlating the output events of the DVSs by using leaky integrate-and-
fire (LIF) neurons after the stereo vergence fusion. The trajectory of each gesture is
automatically spotted by setting the threshold of LIF neurons, and, subsequently, sixteen
feature vectors are extracted from each spotted gesture trajectory. The thresholds of LIF
neurons are adaptively adjusted based on the disparity obtained from the stereovision to
achieve distance invariant performance of gesture spotting. Gesture patterns were classified
by using hidden Markov model (HMM)-based gesture models. The implemented system was
tested with 6 subjects (3 untrained subjects and 3 trained subjects) producing continuous hand

40
gestures (22 trials of 9 successive gestures for each subject). Achieved recognition rates
ranged from 91.9 % to 99.5% depending on subject.

CHAPTER 6

CONCLUSION, LIMITATIONS AND FUTURE WORK

We achieved our objective without any hurdles i.e. the control of a robot using gestures. The
robot is showing proper responses whenever we move our hand. For controlling the robot

41
remotely, Holteks’ encoder-decoder pair (HT12E and HT12D) together with a 433MHz
transmitter-receiver pair is used. HT12E and HT12D are CMOS ICs with working voltage
ranging from 2.4V to 12V. Encoder HT12E has eight address and another four address/data
lines. The data set on these twelve lines (address and address/data lines) is serially
transmitted when transmit-enable pin TE is taken low. The data output appears serially on
DOUT pin. The data is transmitted four times in succession. It consists of differing lengths of
positive-going pulses for ‘1’ and ‘0,’ the pulse width for ‘0’ being twice the pulse-width for
‘1.’ The frequency of these pulses may lie between 1.5 and 7 kHz depending on the resistor
value between OSC1 and OSC2 PINS.

Disadvantages of the Current Gesture

The current gesture recognition system is a vision-based system which has many
disadvantages, Including

 Costly solution
 Need high resolution cameras
 Highly sensitive to noise in image processing (lens aberrations)

Advantages of Touch less Gesture Recognition

The disadvantages of the vision-based recognition system have been overcome in the touch-
less based Gesture Recognition system. The advantages of the touch less sensing system are:

 Cheaper solution
 Easy to develop
 Easy to maintain
 Easy to replace
 Easy to access
 Touch-less

LIMITATIONS AND FUTURE WORK

42
 The on-board batteries occupy a lot of space and are also quite heavy. We can either
use some alternate power source for the batteries or replace the current DC Motors
with ones which require less power.
 Secondly, as we are using RF for wireless transmission, the range is quite limited;
nearly 50-80m. This problem can be solved by utilizing a GSM module for wireless
transmission. The GSM infrastructure is installed almost all over the world. GSM will
not only provide wireless connectivity but also quite a large range.
 Thirdly, an on-board camera can be installed for monitoring the robot from faraway
places. All we need is a wireless camera which will broadcast and a receiver module
which will provide live streaming.

REFERENCES

43
[1] “Gesture Controlled Robot PPT”

<[Link]

[2] “Gesture Controlled Tank Toy User Guide”

<[Link] transmitter>

[3] “Embedded Systems Guide (2002)”

[Link]

rol_interface_for_a _car_robot>

[4] “Robotic Gesture Recognition (1997)” by Jochen Triesch and Christoph Von Der
Malsburg [Link]

[5] “Real-Time Robotic Hand Control Using Hand Gestures” by Jagdish Lal Raheja, Radhey
Shyam, G. Arun Rajsekhar and P. Bhanu Prasad

[6] “Hand Gesture Controlled Robot” by Bhosale Prasad S., Bunage Yogesh B. and Shinde
Swapnil V.

[7]<[Link]

[8]< [Link]

[9] <[Link] [

10] <[Link]

Hand-Control-Using-Hand-Gestures>

[11]<[Link]
[12][Link]
electromotive-force

[12] Matthias Rehm, Nikolaus Bee, Elisabeth André, Wave Like an Egyptian - Accelerometer
Based Gesture Recognition for Culture Specific Interactions, British Computer Society, 2007

[13] ying Wu and Thomas S. Huang, "Vision-Based Gesture Recognition: A Review", In:
Gesture-Based Communication in Human-Computer Interaction, Volume 1739 of Springer
Lecture Notes in Computer Science, pages 103-115, 1999, ISBN 978-3-540-66935-7, doi
10.1007/3-540-46616-9.

44
[14]Alejandro Jaimesa and Nicu Sebe, Multimodal human– computer interaction: A survey,
Computer Vision and Image Understanding Volume 108, Issues 1-2, October– November
2007, Pages 116-134 Special Issue on Vision for Human-Computer Interaction,
doi:10.1016/[Link].2006.10.019

45