PROJECT REPORT

(BTCS 703-18)
On
“Virtual Drawing Tool”

Submitted in partial fulfillment of the `

Requirements for the award of

Degree of Bachelor of Technology in Computer Science & Engineering

Submitted By:
Name: Aaryan Sachdeva
University Roll No. 2002338

Submitted To:
Mr. Rajeev Sharma
Assistant Professor

Department of Computer Science & Technology

Chandigarh Engineering College- CGC, Landran, Mohali
 CERTIFICATE

This is to certify that Mr. Aaryan Sachdeva has completed the Semester Training during the
period from 25thJune,2023 to 27th Nov,2023 in our Organization as a Partial Fulfillment of Degree
of Bachelor of Technology in Computer Science & Engineering.

(Signature of Project supervisor )

Date: 28-12-2023
 DECLARATION

I hereby declare that the Project Report entitled ("Virtual Drawing Tool") is an authentic record of my
own work as requirements of 7th sem academic during the period from 25th June,2023 to 27th Nov,
2023 for the award of degree of B. Tech. ( Computer Science & Engineering , Chandigarh
Engineering College under the guidance of Mr. Rajeev Sharma

Date: 28-12-2023
Aaryan Sachdeva
2002338

Certified that the above statement made by the student is correct to the best of our knowledge and
belief.

Signatures

Examined by

1. 2. 3. 4.

Head of Department
(Signature and Seal)
 ACKNOWLEDGMENT

I take this opportunity to express my sincere gratitude to the Director- Principal Dr. Rajdeep
Singh Chandigarh Engineering College, Landran for providing this opportunity to carry out the
present work.
I am highly grateful to the Dr. Sukhpreet Kaur HOD CSE, Chandigarh Engineering College,
Landran (Mohali). I would like to expresses my gratitude to other faculty members of Computer
Science & Engineering department of CEC, Landran for providing academic inputs, guidance &
Encouragement throughout the training period. The help rendered by Mr. Rajeev Sharma ,
Supervisor for Experimentation is greatly acknowledged. Finally, I express my indebtedness to
all who have directly or indirectly contributed to the successful completion of my software
training

(i)
 TABLE OF CONTENTS

ACKNOWLEDGMENT i
Table of Contents ii
List of Tables iii
List of Figures iv
ABSTRACT v
1. INTRODUCTION 1
1.1 Feasibility Analysis: 3
1.1.1. Proposed system 3
1.1.2. Features Extraction 4
2. SOFTWARE REQUIREMENTS AND SPECIFICATIONS 5
2.1. Analysis document 5
2.2 Literature survey 5
2.3. Overall description 8
2.4. Specific requirements 10
2.5 Requirements 9
3 SYSTEM DESIGN 14
3.1. Writing hand pose detection 14
3.2 Hand region segmentation 14
3.3. Hand centroid localization 15
3.4. Entity relationship diagram for painting 17
3.5. Fingertip tracking 18
3.6. Colour tracking of object at fingertip 18
3.7. Contour detection of the mask of colour object 19
3.8. Drawing the Line using the position of the contour 20
4. SYSTEM IMPLEMENTATION & TESTING 21
4.1. Fingertip Detection Model 22
4.2. Techniques of Fingertip Recognition DatasetCreation: 24
5. IMPLEMENTATION(Modules) 28
5.1 Creating a hand tracking program 30
6. TESTING 36
7 CONCLUSION 42
7.1 Future work 44
8 REFERENCES 45

(ii)
 LIST OF TABLES

Table 1 Various stakeholders involved 10

Table 2 Developer Hardware Requirements 12
Table 3 Developer Software Requirements 12
Table 4 User Hardware Requirements 13

(iii)
 LIST OF FIGURES

Figure 1 System of architecture 3

Figure 2 Silhouette using contour selection Proposed Algorithm 4
Figure 3 Flowchart of Proposed System 14
Figure 4Hand Region Segmentation 15
Figure 5Hand Centroid Localization 16
Figure 6Fingertip Tracking 18
Figure 7Colour Tracking of Object at Fingertip 18
Figure 8 Colour Trackbar 19
Figure 9 Mask Detection 19
Figure 10 Drawing Line using the position of the contour 20
Figure 11 Workflow of the system 21
Figure 12 Video to Images 24
Figure 13 Taking pictures in different backgrounds 25
Figure 14 Word written in air traced on a black image 27
Figure 15 Actual Character vs Trajectory 36

(iv)
 ABSTRACT

As we know, the vision-based technology of hand gesture recognition is an important part of human-
computer interaction (HCI). In the last decades, keyboard and mouse play a significant role in human-
computer interaction. However, owing to the rapid development of hardware and software, new types of
HCI methods have been required. In particular, technologies such as speech recognition and gesture
recognition receive great attention in the field of HCI.

Gesture is a symbol of physical behavior or emotional expression. It includes body gesture and hand
gesture. It falls into two categories: static gesture and dynamic [Link] the former, the posture of the
body or the gesture of the hand denotes a sign. For the latter, the movement of the bodyor the hand
conveys some messages. Gesture can be used as a tool of communication between computerand human
It is greatly different from the traditional hardware based methods and can accomplish human-computer
interaction through gesture recognition. Gesture recognition determines the user intent through the
recognition of the gesture or movement of the body or body parts. In the pastdecades, many researchers
have strived to improve the hand gesture recognition technology. Hand gesture recognition has great value
in many applications such as sign language recognition augmented reality (virtual reality) [sign language
interpreters for the disabled and robot control].

In the authors detect the hand region from input images and then track and analyze the movingpath to
recognize America sign language. In Shimada et al. propose a TV control interface using hand gesture
recognition. Keskin et al. divide the hand into 21 different regions and train a SVM classifier to model
the joint distribution of these regions for various hand gestures so as to classify the gestures. Zeng et al.
improve the medical service through the hand gesture recognition. The HCI recognition system of the
intelligent wheelchair includes five hand gestures and three compound states. Their system performs
reliably in the environment of indoor and outdoor and in the condition of lighting change.

The overview of the hand gesture recognition is described in Figure 1. First, the hand is detected using
the background subtraction method and the result of hand detection is transformed to a binary image.
Then, the fingers and palm are segmented so as to facilitate the finger recognition. Moreover, the fingers
are detected and recognized. Last, hand gestures are recognized using a simple rule classifier.

(v)
 CHAPTER 1 - INTRODUCTION

The work flow of hand gesture recognition is described as follows. First, the hand region is detected from
the original images from the input devices. Then, some kinds of features are extracted to describe hand
gestures. Last, the recognition of hand gestures is accomplished by measuring thesimilarity of the feature
data. The input devices providing the original image information includes normal camera, stereo camera,
and ToF (time of flight) camera. The stereo camera and ToF camera additionally provide the depth
information so it is easy to segment the hand region from the backgroundin terms of the depth map. For
the normal camera, the skin color sensitive to the lighting condition and feature points are combined to
robustly detect and segment the hand region. When the region of interest (ROI, the hand region in the
case) is detected, features are needed to be extracted from the ROI region. Color, brightness, and gradient
values are widely used features. Li and Kitani describe various features for hand region detecting
including the Gabor filter response, HOG, SIFT, BRIEF, and ORB. For the recognition of hand gestures,
various classifiers, for example, SVM (support vector machine), HMM (hidden Markov model), CRF
(conditional random field), and adapted boosting classifier are trained to discriminate hand gestures.
Although the recognition performance of these sophisticated classifiers is good, the time cost is very high.

We present an efficient and effective method for hand gesture recognition. The hand region is detected
through the background subtraction method. Then, the palm and fingers are split so as to recognize the
fingers. After the fingers are recognized, the hand gesture can be classified through a simple rule
classifier.

The novelty of the proposed method is listed as follows.

(i)

The first novelty of the proposed method is that the hand gesture recognition is based on the result of
finger recognition. Therefore, the recognition is accomplished by a simple and efficient rule classifier
instead of the sophisticated but complicated classifiers such as SVM and CRF.

(ii)

Some previous works need the users to wear data glove to acquire hand gesture data. However, the special
sensors of data glove are expensive and hinder its wide application in real life. In the work [25], the
authors use TOF camera, that is, Kinect sensor, to capture the depth of the environment and a special
tape worn across the wrist to detect hand region. Our approach only uses a normal camera to capture the
vision information of the hand gesture meanwhile does not need the help of the special tape to detect hand
regions.

(iii)

1
The third advantage of the proposed method is that it is highly efficient and fit for real-time applications.

However, hand gestures as such are not sufficient for entering text. This requires the development of non-
contact air writing systems that can replace tactile and electromechanical input fields, leading to a more
natural human-computer interaction (HCI) approach.

A vision- based system for recognizing handwriting in the air is not a new problem, and extensive work
has been done over the past two decades. One of the first works by Oka et al. (Oka etal., 2002) used a
sophisticated device with an infrared and color sensor to track and detect trajectories of simple geometric
shapes with their fingertips.

In (Amma et al., 2012), inertial sensors with a glove were used to detect and recognize handwriting from
the air continuously. Recently,Misra et al. (Misra et al., 2017) developed a hand gesture recognition
framework capable of recognizing letters, numbers, arithmetic operators, and 18 printable ASCII
characters using a red marker on the tip of the index finger to identify fingertips.

Despite satisfactory performance in terms of path recognition accuracy, previous approaches using
cumbersome motion detection and detection hardware acknowledge numerous limitations in user
behavior. For example, wearing protective gloves can disrupt the user's natural handwriting pattern andis
often viewed by many users as an unwanted burden.

We present an efficient and effective method for hand gesture recognition. The hand region is detected
through the background subtraction method. Then, the palm and fingers are split so as to recognize the
fingers. After the fingers are recognized, the hand gesture can be classified through a simple rule
classifier.

2
1.1 Feasibility Analysis:
1.1.1. Proposed system

The proposed system can be classified into mainly two steps after acquiring the input image
from camera,videos or even an Object of Interest. These steps are: Extraction Method image
pre- processing and Features estimation and Extraction [1].

Figure 1 System of architecture.

Extraction Method and image pre-processing:

Segmentation process is the first process forrecognizing Object of Interest. It is the process
ofdividing the input image (in this case Object of Interest image) into regionsseparated by
boundaries.
The segmentation process depends on the type of object, if it is dynamic object then the Object
need to be located and tracked , if it is static object with no moving parts such as acoloured
bottle cap or a pen of particular colour the input image have to be segmented only. The object
should be located firstly generally a bounding box is used to specify the dependingon the object
colour and secondly, the object have to be tracked, for tracking the object there are two main
approaches either the video isdivided into frames and each frame have to be processed alone,
in this case the object frame is treated as a moving OBJ(Object-of-Interest) and segmented, or
using some tracking information suchas shape, object colour using some filter. The
commonhelpful cue used for segmenting the object is its colour, since it is easyand invariant to
scale,translation, and rotation changes. In this case a coloured markers which provide exact
informationabout the orientation and position.
The color space used in a specific application plays an essential role in the success of
segmentation process, however colour spaces are sensitive to lighting chan

3
1.1.2. Features Extraction

Good segmentation process leads to perfect features extraction process and the latter playan
important role in a successful recognition process.
The modules of OpenCV for image processing applications are given below:
CORE module contains basic data structures and basic functions used by other modules.
IMGPROC module contains image processing related functions such as linear, non-linear
image filtering and geometrical image transformations etc. VIDEO module contains motion
estimation and object tracking algorithms.
ML module contains machine-learning interfaces. High GUI module contains the basic I/O
interfaces and multi-platform windowing capabilities.
Features vector of the segmented image can be extracted in different ways according to
particular application. Various methods have been applied for representing the features can be
extracted. In this method we have used the shape of the object such as object contour [2]. A
contour is a closed curve of points or line segments that represents the boundaries of an object
in the image. Contours are essentially the shapes of objects in an image. Contours aresometimes
called a collection of points or line segments that overall represent the shape of theobject in an
image.

Figure 2 Silhouette using contour selection Proposed Algorithm

4
 CHAPTER 2 - SOFTWARE REQUIREMENTS AND SPECIFICATIONS

2.1. ANALYSIS DOCUMENT

Flow requirements Analysis results in the specification of software’s operational characteristics.
It indicates software’s interface with other system elements and establishes constraints that the
software must meet. It allows the software engineer to elaborate on basic requirements
established during earlier requirement engineering tasks and builds models that depict user
scenarios, functional activities, problem classes and their relationships, system and class
behavior.
The analysis model must achieve three primary objectives:
1. To describe what the customer wants,
2. To establish a basis for the creation of a software design,
3. To define a set of requirements that can be validated once the software is build.

One view of analyzing modelling, called structured analysis, considers data and the processes
that transform the data as separate entities. Data objects are modelled in way that defines their
attributes and relationships. Processes that manipulate objects are modelled in a manner that
shows how they transform data as objects flow through the system.
A second approach to analysis modelling, called object-oriented analysis, focuses on the
definition of classes and the manner in which they collaborate with one another to effect the
customer requirements UML and unified processes are predominately object-oriented.
To solve actual problems in an industry, a software engineer or a team of engineers, must
incorporate a development strategy that encompasses the process, methods and tools and layers.
This strategy is often referred to as a process model. Process model is chosen based on the
nature of the project and application, the methods and tools to be used.
The various process models are:
1. Waterfall Model
2. Prototype Model
3. RAD Model
4. Incremental Model

2.2 LITERATURE SURVEY

Automatic object tracking has many uses in computing, such as Computer vision and human-
machine interaction. Various applications of tracking algorithms are suggested in

5
The hand gesture recognition[6-7], another group for text tracking and recognition [8, 18], and
tomonitorthe body movement of objects for virtual reality [10] and character recognition based
on fingertracking [19- 21], etc. Bragatto et al. developed a method that automatically translates
Brazilian Sign Languagefrom video input. They used a multilayer NN (Neural Perceptron)
network witha piecewise linearapproximate trigger function for real-time video processing. This
activation function reduces the mean complexity time of NN. In addition, they usedNN in two
stages: color recognition and steps to assess hand posture. Their results show that theproposed
methodworks well with a detection rate of99.2% [5]. Cooper also introduced a method for
managing 3D cell bioprinting that is more complex than the generalized set. Cooper developed a
technique that reduces tracing by identifying errors in histhesis's classification and tracing
processes. Cooper used two pretreatment steps; One isformovement, and the other is used to
determine theshape of the hand. He also used the screen to expandhis vocabulary. The viseme
is an essentialposition of the mouth and face in the pronunciation of a phoneme and the visual
representationof phonemes. Over time, he develops a poorly structured learning method that
identifies characters.

Araga et al. proposed a hand gesture recognition system using Jordan's Recurrent Neural
Network (JRNN). Their system modeled the 5 and 9 differenthand positions through a sequence
of representativestatic images. He then takes the footage as input andbegins to classify the hand
poses. JRNN finds the input gesture after the temporal behavior of theposture sequence has been
recognized. In addition, they are developing a new training method. The proposed methodshows
a precision of 99.0% for five different hand positions, while it achieves an accuracy of 94.3%
for nine gestures [6]. In [7], Yanget al .; discussed an alternative solution to matchinga sequence
of images to a pattern, which usually occurs with hand gesture recognition. The proposedmethod
is not based on skin color patterns and can work even with poor [Link] combined
both the segmentation process and recognition usinga cross-cluster process. Their results show
better performance with a performance loss of 5% for bothmodels. Neumannet al. developed a
method to findand recognize text in authentic images. In their article, they used a hypothesis
framework that can handle multiple lines of text. They also use synthetic characters to train the
algorithm, and, finally, they use maximum stable areas (MSER), which offerrobustness to
geometric and lighting conditions [8].

Furthermore, Wang et al. discussed the color-based motion detection system for indoor and
outdoorenvironments. In the proposed method, they used a webcam and a colored T-shirt to

6
 for virtual realityapplications. Jari Hannuksela et al. [19], ToshioAsano et al. [20], and Sharad
Vikram et al. [21] havecharacter recognition systems based on finger tracing. In [19], the author
presents a motion-based tracking algorithm that combines two Kalman filteringtechniques and
expectation maximization (EM) methods to estimate two different motions. Finger and
cameramovement. The estimate is basedon the motion properties that the scene calculates for
each image. Its main idea is to control mobile devices by simply swiping a fingerin front of a
camera. In [20], the authors discuss a visual interface that recognizes Japanese katakana
characters in the air. To follow the movement of the hand, they used the LED pen and a camera.
They convert the signal of the pencil into direction codes. Codes are normalizedto 100 data
items to eliminate the effectof typing speed, where 46 Japanese characters are defined. For a
single camera, they achieve a character recognition accuracy of 92.9%, and for multiple
cameras, they accomplisha gesture- directing accuracy of 9°.
● Robust Hand Recognition with Kinect Sensor

In [3], the system proposed used the depth and colour information from the Kinect sensor to
detect the hand shape. As for gesture recognition, even with the Kinect sensor. It is still a very
challenging problem. The resolution of this Kinect sensor is only 640×480. It works well to
track a large object, e.g., the human body. But following a tiny thing like a finger is complex.
● LED fitted finger movements

Authors in [4] suggested a method in which an LED is mounted on the user's finger, and the
web camera is usedto track the finger. The character drawn is compared with that present in
the database. It returns the alphabet that matches the pattern drawn. It requires a red-coloured
LED pointed light source is attached to the finger. Also, it is assumed that there is no red-
coloured object other than the LED light within the web camera's focus.
● Augmented Desk Interface

In [5] Augmented segmented desk interface approach for interaction was proposed. This
system makes use and a video projector and charge-coupled device (CCD) camera so that
using the fingertip; users can operate desktopapplications. In this system, each hand performs
different tasks. The left hand is used to select radial menus.

7
2.3. OVERALL DESCRIPTION

● To create a virtual canvas to sketch.

● To detect the human finger as a colour marker.

● To do the morphological operations.

● To create an interface between user and the system.

● To ensure that, the interface is very simple and easilyunderstandable by the user.

● The user should be able to draw what he wishes todraw without any interruptions.

● In future, this is useful for making kids to learndrawing in schools in an interactive

way.

The overall description of the proposed system encompasses a multifaceted approach

aimed at creating an interactive and user-friendly virtual canvas for sketching. The
key objectives and functionalities of the system are outlined as follows,
emphasizing its potential applications, simplicity, and user-centric design.
1. Virtual Canvas Creation:

- The primary goal is to establish a digital platform or virtual canvas that enables
users to engage in sketching activities. This virtual canvas serves as an innovative
space for creative expression, providing users with a modern and dynamic
medium for their artistic endeavors.

2. Human Finger Detection as a Color Marker:

- The system integrates a sophisticated mechanism to detect the human finger as a

color marker. This entails leveraging computer vision techniques, such as image
processing and color detection algorithms, to precisely identify and track the
movement of the user's finger. The finger serves as a natural and intuitive tool for
drawing on the virtual canvas.

3. Morphological Operations:

- Morphological operations are applied to enhance the accuracy and stability of the
finger detection process. Techniques such as erosion and dilation are employed to
refine the binary image and ensure a smooth representation of the user's finger
movements on the virtual canvas. These operations contribute to the overall
robustness of the system.

8
4. User-System Interface:

- A crucial aspect of the system is the creation of a seamless interface between the
user and the digital canvas. The interface facilitates user interaction, allowing
individuals to draw, sketch, and express their creativity effortlessly. Design
considerations prioritize simplicity and intuitiveness, ensuring that users can
easily navigate and engage with the system.

5. Simplicity and User-Friendliness:

- Emphasis is placed on ensuring that the user interface is exceptionally simple and
easily understandable. The system is designed with the user in mind, prioritizing
an intuitive experience that minimizes any learning curve. This approach
encourages users of all skill levels to engage with the virtual canvas effortlessly,
fostering a positive and enjoyable creative process.

6. Uninterrupted Drawing Experience:

- One of the core objectives is to provide users with an uninterrupted drawing

experience. The system strives to eliminate any hindrances or disruptions,
allowing individuals to focus on their artistic expression without technical
distractions. This emphasis on a smooth and seamless drawing experience
enhances user satisfaction.

7. Educational Potential for Kids:

- Looking towards the future, the system holds significant potential for educational
applications, particularly in schools. The interactive nature of the virtual canvas
can be leveraged as a valuable tool for teaching and learning drawing skills among
children. This innovative approach aligns with modern educational paradigms,
offering a dynamic and engaging platform for kids to explore and nurture their
artistic talents.

In summary, the overall description envisions a versatile and user-centric system

that not only facilitates creative expression on a virtual canvas but also holds
promise for educational applications. The careful consideration of simplicity,
user-friendliness, and uninterrupted drawing experiences reflects a commitment to
providing a delightful and accessible platform for users of all ages and skill levels.
9
 [Link] REQUIREMENTS
In the open domain dynamic QA system, we have pointed out three kinds of stakeholders:
Builder, User and Tester. Builder and Tester also have different roles. We define the
Table 1 Various stakeholders involved

Stakeholder Role Task

Builder Software engineer Building the QA system:

Building the question database,

answering database, and result matching
database.
Maintaining and updating the database.
Designing the algorithm of the QA system.
Human Expert Answering questions and producing knowledge.
Providing the answers of the questions which users
put into the question database.
(So in other words, Experts are the builders of
answering database.)

10
 User Main User Consuming knowledge and
producing questions.
Receiving the answers that they asked from the
answering database.
Putting questions into the questions database. (In
other words, Main User is also the builder
of question database.)
Tester System Tester Testing the validation and the effectiveness of

the system.
Quality Assurance Checking the validation of information, (both
Manager questions and knowledge). Moving the invalid
information from the database.

2.4.1 Overview of Functional Requirements

Purpose: To retrieve answers to the users from the
web.
Inputs: Natural Language Query from the real world entity.
Outputs: Answers to the users question.
2.4.2 Overview of Data Requirements
The user should contain an internet connection to get the output.
User View of Product Use
- Education
- Medical
- Social
- Information Retrieval
-

2.5 REQUIREMENTS
2.5.1. FACILITIES REQUIRED FOR THE PROJECT REPORT
- DEVELOPER HARDWARE REQUIREMENTS
The project utilizes the hardware resources which are easily available and can be easily utilized
to find the correct balance between the two and this is the whole point of the thing being carried
out to the extent of oblivion. More importantly the project needs to use GPU’s

11
locally on your machine the minimum system requirements are –
Table 2 Developer Hardware Requirements
[Link] Component Minimum Requirement
1. Processor 64 bit, Quad-Core, 3.50 GHz minimum Clock speed
2. RAM 16 GB

3. Graphics Card NVIDIA GTX 1650 and Above

1. OS Windows 10 or above
2. HDD Min 256 Gb
3. SSD The more the merrier (greatly improves performance of the
retriever)

[Link] DEVELOPER SOFTWARE Requirements

[Link] Component Version
1. Python 3.9 or above
2. Docker 4.5.1 or above

3. Elasticsearch 7.17.0

4. VS Code 1.64.2
5. Notion 1.24.1

6. Streamlit Latest from pip

[Link] USER HARDWARE REQUIREMENTS

Table 4 USER HARDWARE REQUIREMENTS

[Link] Component Minimum Requirement

1. Processor 64 bit, Quad-Core, 2GHz minimum Clock speed
2. RAM 8 GB
3. Graphics Card NA

12
 4. OS Windows 10 or above
5. HDD Min 256 Gb
6. SSD Not Required

(ii) Inner Circle of the Maximal Radius. When the palm point is found, it can draw a circle with the
palm point as the center point inside the palm. The circle is called the inner circle because it is included
inside the palm. The radius of the circle gradually increases until it reaches the edge of the palm. That is
the radius of the circle stops to increase when the black pixels are included in the circle. The circle is
the inner circle of the maximal radius which is drawn as the circle with the red color in Figure 6.

(iii) Wrist Points and Palm Mask. When the radius of the maximal inner circle is acquired, a larger
circle the radius of which is 1.2 times of that of the maximal inner circle is produced. The circle is
drawn as the blue color circle in Figure 6. Then, some points are sampled uniformly along the circle.
That is,

where is the position of the palm point, is the radius of the circle, and is the sampling step.

For each sampled point on the circle, its nearest boundary point is found and lined to it. The boundary
point is judged in a simple way. If the 8 neighbors of a pixel consist of white and black pixels, it is
labeled as a boundary point. All of the nearest boundary points found are linked to yield the palm mask
that can be used to segment fingers and the palm. The method for searching the palm mask is described
in Algorithm 1. The palm mask of the hand image of Figure 3 is demonstrated in Figure 7. A larger
circle instead of the maximal inner circle is used so as to yield a more accurate palm mask for the
following segmentation.

13
 CHAPTER 3 - SYSTEM DESIGN

3.1 Writing Hand Pose Detection

Recognizing the position of the writing hand and recognizing it through other gestures is an
essential step in initializing aerial writing. Unlike traditional writing, when the pen moves
down, and the pen moves up, writing in the air is not outlined as a writing sequence. Events.
The system recognizes theposition of a writing hand and distinguishes it from a non-writing
hand by counting the number of raisedfingers.

Figure 3 Flowchart of Proposed System

3.2 Hand Region Segmentation

Once we have accurately captured the hand using theabove technique, the segmentation of the
hand area is done using a two-step approach, viz. The skin segmentation and the subtraction of
the backgroundand the final binary image of the hand are obtained as an aggregation of the two.
The proposedalgorithm works well in real-time and provides relatively accurate segmentation
results. Although skin colors vary greatly from breed to breed, it has been observed that skin
color has a small area between different skin types, while skin luminosity differs significantly.

14
 Figure 4Hand Region Segmentation

Background subtraction: Since accurate hand detection with the Faster R-CNN handheld
detectorfollowed by filtering of skin color at the boundary ofthe candidate's hand provides a
reasonably goodsegmentation result, the subtraction step of background is only used to remove
skin-colored

objects (not part of the hand) that are in the bounding box of the recognized hand may be present.

3.2 Hand Centroid Localization

In the process of hand centroid localization within the broader scope of computer vision
applications, precision in determining the exact center of gravity of the hand is pivotal. To
achieve this, a sophisticated system employs two distinct algorithms for generating initial
estimates of the center of gravity. The ultimate accuracy is attained by calculating the average
of these two estimates. This approach ensures robustness and reliability in subsequent steps of
the system.

The first algorithm leverages the distance transformation method to obtain the initial estimate
of the center of gravity, denoted as (xc1, yc1). The distance transform image is a key element
in this method, where each pixel is represented by its distance from the nearest edge pixel. The
Euclidean distance metric is employed to measure the distance between a given pixel and its
closest edge pixel. Consequently, the pixel with the highest intensity in the distance transform
image is identified as the preliminary center of gravity.

This distance transformation method serves as an efficient means to capture the spatial
distribution of the hand, allowing the system to discern the central point with a high degree of
accuracy. The utilization of Euclidean distance ensures that the measured distances are
representative of the true geometric relationships between pixels and edges, contributing to the

15
reliability of the initial estimate.

In conjunction with the distance transformation method, the system incorporates a second
algorithm to derive another initial estimate of the center of gravity. The combination of these
two algorithms enriches the system's capability to handle diverse hand configurations and
varying lighting conditions, promoting adaptability and robustness.

The final center of gravity is determined by taking the average of the estimates generated by
the two algorithms. This fusion of information enhances the overall accuracy of the centroid
localization process, mitigating potential errors associated with individual algorithms and
providing a more stable and dependable outcome.

In summary, the hand centroid localization process involves a dual-algorithmic approach,

employing the distance transformation method and a complementary algorithm. By
strategically combining their outputs, the system ensures a precise and reliable determination
of the center of gravity, laying a solid foundation for subsequent stages in computer vision
applications related to hand tracking, gesture recognition, and human-computer interaction.
This meticulous approach contributes to the robustness and adaptability of the overall system,
addressing challenges associated with varying hand poses and environmental conditions.

Since determining the exact center of gravity of the hand is critical in the following steps, the
system uses two algorithms to determine the initial estimates of the center of gravity, and the
final centerof gravity is calculated as the average of the two. The distance transformation
method is used to obtain thefirst estimate of the center of gravity (xc1, yc1). In the distance
transform image, each pixel is represented by its distance from the next edge pixel. Euclidean
distance wasused to measure the distance between a pixel and its closest edge pixel. Therefore,
the pixelwith the highest intensity in the distance transform image is taken as the
center of gravity.

Figure 5Hand Centroid Localization

16
3.4. ENTITY RELATIONSHIP DIAGRAM FOR PAINTING

17
 3.5 Fingertip Tracking

Hand detection and tracking on consecutive images is an essential part of fingertip detection
and tracking. Experiments show that using the Faster R-CNN handheld detector for each frame
is computationally intensive and produces frame rates well below real-time performance.
Therefore, the KCF tracking algorithm (Henriques et al., 2015) is used to track the area of the
captured hand.

Figure 6Fingertip Tracking

3.6 Colour Tracking of Object at fingertip

Incoming webcam images must be converted toHSV color space to recognize hand-colored
objects.

Figure 7Colour Tracking of Object at Fingertip

18
The algorithm code converts the incoming image into HSV space, which is a very suitable
color spaceand perfect for color tracking.

Figure 8 Colour Trackbar

The tracking bars have organized the HSV values into the required color range of the colored
object you placed on your finger.

3.7 Contour Detection of the Mask of Colour Object

After detecting the Mask in Air Canvas, it is now time to locate its center position for drawing
the Line. The system will perform some morphologicaloperations on the Mask to make it free
of impuritiesand detect contour easily.

Figure 9 Mask Detection

19
 3.8 Drawing the Line using the position of the contour

The actual logic behind this computer vision project is to create a Python deque (a data
structure). The deque will memorize the position of the outline in each subsequent frame, and
we will use these accumulated points to create a line using OpenCV'sdrawing capabilities.

Now use the outline position to decide whether to click a button or draw on the provided sheet.
Some of the buttons are at the top of the canvas. When thepointer enters this area, it is activated
according to the method present in this area.

Figure 10 Drawing Line using the position of the contour

Object position: Image extracted from the videosequence. Extract the color image from the
reference: This proposed method tracks the index finger movement, colored blue. We don't
have a reference image, so any previous image is a reference to the next. Now take the
difference in theimages and extract the color and movement of the object.

Edge Enhancement (EE) - Edge enhancementtechnique makes the object location algorithm
robust against noise, different lighting conditions, darkening, and fading of objects, even in
low contrast image

20
 CHAPTER 4 - SYSTEM IMPLEMENTATION & TESTING

The frames are read and convert the captured frames to HSV color space (Easy for
color detection).

Make the canvas frame and put the respective link buttons on it.

Now, Set the track bar values for finding the mask of the colored marker.

Preprocessing of the mask is done with morphologicaloperations (Eroding & Dilation).

The next step goes on like this by, Detecting thecontours, finding the center coordinates of
large contour and keep storing them in the array for next frames (Arrays for drawing points on
the canvass).

Finally, draw the points stored in an array on the frames and canvas.

Figure 11 Workflow of the system

This system needs a dataset for the Fingertip Detection Model. The Fingertip Model's
primarypurpose is used to record the motion, i.e., the air character.

21
 4.1 Fingertip Detection Model:

Air writing can be merely achieved using a stylus or air- pens that have a unique colour [2].
The system, though, makes use of fingertip. We believe people should be able to write in the
air without the pain of carrying a stylus. We have used Deep Learning algorithms to detect
fingertip in every frame, generating a list of coordinates.

stylus. Instead of relying on a stylus or air pen with a unique color, this system utilizes the
fingertip as the writing tool. The motivation behind this approach is to eliminate the
inconvenience of carrying a stylus, making the process more intuitive and accessible.

To implement this system, the developers have employed Deep Learning algorithms. Deep
Learning is a subset of machine learning that involves neural networks with multiple layers
(deep neural networks). In the context of this system, the Deep Learning algorithms are used
to detect the fingertip in each frame of a video or image sequence.

Here is a step-by-step explanation:

[Link] Collection:

Gather a dataset containing images or frames of people writing in the air using their
fingertips.

[Link] Preprocessing:

Preprocess the collected data, which may involve tasks like resizing images, normalizing
pixel values, or augmenting the dataset to improve model generalization.

[Link] Learning Model Training:

Train a Deep Learning model, such as a Convolutional Neural Network (CNN) or a

combination of various neural network architectures, to recognize and detect fingertips in the
given images or frames.

[Link] Detection:

Apply the trained model to each frame of the video or image sequence in real-time. The
model will output a list of coordinates corresponding to the detected fingertip(s) in each
frame.

[Link] Generation:

22
Generate a list of coordinates based on the detected fingertip positions. This list represents the
trajectory or path of the fingertip as it moves in the air.

[Link] Writing Recognition:

Use the generated list of coordinates to recognize the written characters or shapes. This
recognition process may involve further machine learning techniques or algorithms depending
on the complexity of the desired output.

[Link] Visualization:

Visualize the recognized air writing, potentially by displaying it on a screen or converting it

into a digital format.

By employing Deep Learning algorithms, the system is capable of dynamically detecting and
tracking the fingertip movement, allowing users to write in the air without the need for a
physical writing instrument. The generated list of coordinates provides the spatial information
needed to interpret and visualize the air writing.

The system for air writing described in this context offers a unique and intuitive approach,
allowing users to write in the air without the need for a stylus or specialized pen. Unlike
traditional methods that rely on accessories with distinct colors, this system utilizes fingertip
detection through the application of Deep Learning algorithms. The motivation behind this
innovation is to enhance user convenience by eliminating the requirement to carry a physical
stylus. The development process involves collecting a dataset of images or frames capturing
individuals writing in the air with their fingertips. Following data preprocessing steps, a Deep
Learning model, potentially a Convolutional Neural Network (CNN), is trained to recognize
and detect fingertips in real-time. The output of the model comprises a list of coordinates
representing the detected fingertip positions in each frame. These coordinates form the basis
for generating a trajectory or path of the fingertip's movement. The system can further
recognize the air writing by interpreting the list of coordinates and may employ additional
machine learning techniques for character or shape recognition. The final output can be
visualized on a screen or converted into a digital format, providing a seamless and innovative
means of air writing.

23
 [Link] of Fingertip Recognition DatasetCreation:

Video to Images: In this approach, two-second videos of a person's hand motion were
captured in different environments. These videos were then broken into 30 separate images,
as shown in Figure 3. We collected 2000 images in total. This dataset was labeled manually
using LabelImg[13]. The best model trained on this dataset yielded an accuracy of 99%.

Figure 12 Video to Images

However, since the generated 30 images were from the same video and the same
environment, the dataset was monotonous. Hence, the model didn't work well for discrete
backgrounds from the ones in the dataset.

a. Take Pictures in Distinct Backgrounds: To overcome the drawback caused by the lack of
diversity in the previous method, we created a new dataset. This time, we were aware that we
needed some gestures to control the system. So, we collected the four distinct hand poses, shown
in Figure

24
The idea was to make the model capable of efficiently recognizing the fingertips of all four
fingers. This would allow the user to control the system using the number of fingers he
shows. He or she could now - promptly write by showing one index finger, convert this
writing motion to e-text by offering two fingers, add space by showing three fingers, hit
backspace by showing five fingers, inter prediction mode by showing four fingers, and then
the show 1,2,3 fingers to select the 1st, 2nd or 3rd prediction respectively. To get out of
prediction mode, show five fingers. This dataset consisted of 1800 images. Using a script, the
previously trained model was made to auto- label this dataset. Then we corrected the
mislabelled images and introduced another model. A 94% accuracy was achieved. Contrary
to the former one, this model worked well in different backgrounds.

Figure 13 Taking pictures in different backgrounds

[Link] Recognition Model Training:

In the pursuit of achieving accurate fingertip recognition within our computer vision
framework, a crucial phase involves training a specialized model. Once our meticulously
prepared dataset, enriched with proper labeling, was at our disposal, we proceeded to split it
into distinct training and development sets, with an 85%-15% division ratio. This strategic
division ensures a robust model that generalizes well to unseen data.

For the model architecture, we explored and implemented two renowned pre-trained models,
namely Single Shot Detector (SSD) and Faster Region-based Convolutional Neural Network
(Faster RCNN). Both models are widely recognized in the field of object detection, each with
its unique characteristics. In our experimentation, we found that Faster RCNN outperformed
25
SSD in terms of accuracy, a critical factor in the context of fingertip recognition.

SSD is known for its efficiency in real-time object detection by combining two key
components: one proposing regions and the other classifying them. This simultaneous
detection of objects in a single shot significantly enhances performance and is particularly
suitable for applications requiring real-time responsiveness. It has proven to be a valuable
choice in scenarios where quick and accurate object detection is paramount.

On the other hand, Faster RCNN operates by employing an output feature map from Fast
RCNN to generate region proposals. These proposals undergo evaluation by a Region
Proposal Network before being passed to a Region of Interest pooling layer. The final step
involves presenting the results to two fully connected layers responsible for classification and
bounding box regression. This multi-stage process, although computationally intensive, has
demonstrated superior accuracy in our fingertip recognition task.

To tailor Faster RCNN to our specific recognition needs, we focused on fine-tuning the last
fully connected layer. This layer, critical for recognizing intricate fingertip patterns within the
images, underwent adjustments to optimize the model's performance. The tuned layer plays a
pivotal role in the network's ability to accurately classify and locate fingertips in diverse
visual contexts.

In conclusion, the Fingertip Recognition Model Training phase integrates state-of-the-art

models, SSD and Faster RCNN, into our computer vision pipeline. The preference for Faster
RCNN, given its superior accuracy, underscores our commitment to achieving precise
fingertip recognition. The meticulous tuning of the model, particularly in the last fully
connected layer, fine-tunes its ability to discern fingertip features, ensuring the success of
subsequent stages in our computer vision application. This approach reflects a careful balance
between model complexity and accuracy, crucial for effective fingertip recognition in diverse
real-world [Link] the dataset was ready and labeled, it is divided into train and dev
sets (85%-15%). We used Single Shot Detector (SSD) and Faster RCNN pre-trained models
to train our dataset. Faster RCNN was much better in terms of accuracy as compared to SSD.
Please refer to the Results Section for more information. SSDs combine two standard object
detection modules –one which proposes regions and the other which classifies them. This
speeds up the performance as objects are detected in a single shot. It is commonly used for
real-time object detections. Faster RCNN uses an output feature map from Fast RCNNto
compute region proposals

26
The result is finally given to two fully connected layers for classification and bounding
boxregression [15]. We tuned the last fully connected layer ofFaster RCNN to recognize the
fingertip in the image.

Figure 14 Word written in air traced on a black image

27
 CHAPTER 5 – IMPLEMENTATION (Modules)

In this machine learning application, we have developed code by using Python programming
language along with OpenCV library. Main idea behind this algorithm is to use live feed from
camera and process each frame. However, the algorithms will be implemented on defined ROI
(region of interest).

Step 1: Initialize Some Stuff

Firstly, we import the necessary libraries. Follow ingare mentioned in the image:

Then we initialize variables that are used in the following steps. We can take any colour in this
case we have taken blue colour to be traced on the Contour-Of-Interest. The blue Lower and
the blue Upper NumPy arrays help us in finding the blue coloured cap. The kernal helpsin
smoothing blue cap once found. The b points, g points ,r points and y points deques a reusedto
store the points drawn on the screen of colour blue, green, red, and yellow respectively.

Step 2: Setup the Paint Interface

Now we manually set the coordinates of each of the colour boxes on the frame. We use the
OpenCV function [Link]() to draw the boxes.

Step 3: Start Reading the Video (Frame by Frame) Now we use the OpenCV function
[Link]() method to read a video, frame by frame (using a while loop), either from
a video file or from a webcam in real-time. In this case, we pass 0 to the method to readfrom a
webcam. We can just add the exact same paint interface for ease of usage.

Step 4: Find the Contour-Of-Interest

Once we start reading the webcam feed, we constantly look for a blue colour object in the
frames with the help of [Link]() method and use the blue Upper and blue Lower
variablesinitialized in Step 0. Once we find the contour, we do a series of image operations and
make it smooth. These operations are specific type of morphing techniques as defined below.

28
The machine learning application, implemented in Python with the OpenCV library, processes
live camera feed to facilitate real-time drawing on a predefined Region of Interest (ROI). By
targeting a specific blue color using Numpy arrays, the algorithm detects and tracks the blue-
colored object in the video frames. The paint interface is established with manually set
coordinates for color boxes, allowing users to interactively draw with blue, green, red, and
yellow. The video feed is continuously analyzed, and contours of interest are identified through
morphological operations, creating a smooth and visually appealing drawing experience. The
technical elements, including Numpy arrays, morphological techniques, and OpenCV
functions, converge to form an interactive and dynamic system for real-time color-based
drawing on a live webcam feed.

In this machine learning application, the core functionality is centered around real-time color-
based drawing on a live webcam feed, implemented using Python and the OpenCV library. The
primary objective is to create an interactive and dynamic painting interface with a focus on a
specific Region of Interest (ROI).

The initialization phase involves importing necessary libraries and setting up variables (e.g.,
blue Lower and blue Upper Numpy arrays) to identify a distinct blue color for tracing on the
Contour-of-Interest. These variables, along with a kernel for smoothing, play a crucial role in
detecting and tracking the blue-colored object in the video frames.

The paint interface is established by manually setting coordinates for color boxes using the
OpenCV function [Link](). This creates an intuitive visual interface that enables users
to draw in real-time using different colors: blue, green, red, and yellow.

The application reads the video feed frame by frame using the [Link]() method. By
continuously processing each frame in a while loop, the algorithm integrates the paint interface
seamlessly, providing users with an interactive canvas. This setup allows for ease of usage,
whether drawing from a webcam in real-time or processing video files.

The core of the application lies in the process of finding the Contour-of-Interest within the
webcam feed. It utilizes the [Link]() method, referencing the blue Upper and blue Lower
variables, to identify the blue-colored object. Once the contour is detected, a series of image
operations are applied to enhance its appearance, employing specific morphological
29
 5.1 Creating a hand tracking program

Before we jump into coding, let us discuss how MediaPipe performs hand tracking. Hand
tracking using MediaPipe involves two stages:

● Palm detection - MediaPipe works on the complete input image and provides a cropped
image of the hand.
● Hand landmarks identification - MediaPipe finds the 21 hand landmarks on the cropped
image of the hand.

The 21 hand points that MediaPipe identifies are shown in the image above section:

Image Source: MediaPipe

The image above shows the hand landmarks that MediaPipe uses to identify the hand. The
numbered parts are the hand points.

Coding

Pycharm creates a [Link] file for you automatically after you create a new project. This is where
we will write our code.

Step 1 - Importations and initializations

We start by importing the two libraries we discussed. Importing the libraries enables us to use
its dependencies.

30
We will then create an object cap for video capturing. We require the other three objects to
manipulate our input using MediaPipe:

Step 2 - Capturing an image input and processing it

The code below takes the image input from the webcam. It then converts the image from
BGR to RGB. This is because MediaPipe only works with RGB images, not BGR.

It then processes the RGB image to identify the hands in the image:

Step 3 - Working with each hand

In the code above, we use the if statement to check whether a hand is detected. We then use the
first for loop to enable us work with one hand at a time.

The second for loop helps us get the hand landmark information which will give us the x and
y coordinates of each listed point in the hand landmark diagram. This loop will also give us the
id of each point.
31
We will then find the height, width, and channel of our image using the [Link] function.
We finally get the central positions of the identified hand points.

Step 4 - Drawing the hand landmarks and hand connections on the hand image

In the code above, we circle the hand point number 20. This is the tip of the pinkie finger.

Feel free to use the number of the hand point you want to circle as they are listed on the hand
landmark diagram. We then draw the hand landmarks and the connections between them on the
input image.

Step 5 - Displaying the output

We use the code above to display the output to the user. The output is a real-time video of the
user. It has the user’s hands tracked, hand landmarks, and connections drawn on the hands.

The output of this code is shown in the results section.

Creating a module from our code

Create a new file and name it handTrackingModule. Feel free to name it whatever name you
want. Let us now create a module from the code above by following the steps below:

Step 1 - Importing the required libraries

We first import the Python libraries that we need in our project. This will enable us to use its
dependencies:

32
Step 2 - Creating a class that we will use for hand detection

In the code above, we create a class that we will use for tracking. We then key in the basic
parameters that we require for the hands function to work. MediaPipe provides these parameters
in the hands function.

Afterwards, we provide all the initializations that we need for our class. These are theparameters
above and the MediaPipe initializations.

We put self before every object to allow access to the methods and the attributes of that object.
This in turn allows each object to possess its own attributes and methods.

Step 3 - Creating a method that will track the hands in our input image

33
In the code above, we created a method that we will use to specifically track the hands in our
input image. The code that goes in this method is the one that converts the image to RGB and
processes the RGB image to locate the hands.

It also draws the hand landmarks on the image and finally draws the hand connections.

Step 4 - Creating a method to find the ‘x’ and ‘y’ coordinates of each hand point

In the code above, we created a method that we will use to find the x and y coordinates of each
of the 21 hand points. We also created a list that we will use to store the values of these
coordinates.

The code that goes in this method is the one we use to find the id and hand landmark information
of each hand point. We also put in the code that we will use to circle the hand-point that we
want to use.

Step 5 - Creating the main method

30

34
The code above represents the dummy code that we will use to showcase what the module can
do. In our case, it can identify and track hands. It uses the image and lmlist object. This code
appears in the main method.

Step 6 - Executing the main method

This code implies that, if we are running the module script, then execute the main method.

35
 CHAPTER 6 – TESTING

Fingertipdetection

The existing system only works with your fingers, and there are no highlighters, paints, or
relatives. Identifying and characterizing an object such as a finger from an RGB image without
a depth sensor is a great challenge.
Lack of pen up and pen down motion
The system uses a single RGB camera to write from above. Since depth sensing is not possible,
up and down pen movements cannot be followed.
Therefore, the fingertip's entire trajectory is traced, and the resulting image would be absurd
and not recognized by the model. The difference between hand written and air written ‘G’ is
shown in Figure 1.

Figure 15 Actual Character vs Trajectory

Controlling the real-time system

Using real-time hand gestures to change the system from one state to another requires a lot of
code care. Also, the user must know many movements to control his planadequately.

● The most exciting part of this system is the workflow phase. Writing involves a lot of
functionalities. So, the number of gestures used for controlling this particular system is equal
to these number of actions involved. The basic functionalities that are included in thissystem
are:

Writing Mode - In this state, the system will tracethe fingertip coordinates and stores them.

Colour Mode – The user can change the color ofthe text among the various available colors.

Backspace - Say if the user goes wrong, we needa gesture to add a quick backspace.

36
SCREENSHOTS

37
38
39
40
41
 CHAPTER 7 – CONCLUSION

Demonstration of the image processing capabilities of OpenCV. The ultimate goal is to create
a computer vision machine learning application that promotes Human computer interaction
(HCI) also named Man-Machine Interaction (MMI)] refers to the relation between the human
and the computer or more precisely the machine, and since the machine is insignificant without
suitable utilize by the human there are two main characteristics should be deemed when
designing a HCI system as mentioned :functionality and usability[5]. System functionality
referred to the set of functions or services that the system equips to the users while system
usability referred to the level and scope that the system can operate and performspecific user
purposes efficiently.

The system has the potential to challenge traditional writing methods. It eradicates the need
to carry a mobile phone in hand to jot down notes, providing a simple on- the-go way to do
the same. It will also serve a great purpose in helping especially abled people communicate
easily. Even senior citizens or people who find it difficult to use keyboards will able to use
system effortlessly. Extending the functionality, system can also be used to control IoT devices
shortly. Drawing in the air can also bemade possible. The system will be an excellentsoftware
for smart wearables using which people could better interact with the digital world.
Augmented Reality can make text come alive. There are some limitations of the system which
can be improved in the future. Firstly, using a handwriting recognizer in place of a character
recognizer will allow the user to write word by word, making writing faster. Secondly, hand-
gestures with a pause can be used to control the real-time system as done by
[1] instead of using the number of fingertips. Thirdly, our system sometimes recognizes
fingertips in the background and changes their state. Air-writing systems should only obey
their master's control gestures and should not be misled by people around. Also, we used the
EMNIST dataset, which is not a proper air-character dataset. Upcoming object detection
algorithms such as YOLO v3 can improve fingertip recognition accuracy and speed. In the
future, advances in Artificial Intelligencewill enhance the efficiency of air-writing.
This project makes the user to have an interactive environment where the user can draw
whatever he wants by choosing his required colors from the displayed ones. So, weconclude
that Virtual Sketch is developed using the library NumPy and in Open CV where we have many
libraries and algorithm in built which makes the interfaces more active while using .
We used python as, it have many inbuilt libraries and many modules which represent the
imagination virtually when used along with OpenCV as well as its morphological processes.

42
The image processing capabilities of OpenCV serve as a cornerstone in the development of a
computer vision machine learning application that aims to enhance Human-Computer
Interaction (HCI), also known as Man-Machine Interaction (MMI). HCI revolves around the
relationship between humans and machines, emphasizing the importance of creating systems
that are both functional and usable.

In the context of image processing, OpenCV provides a robust set of tools and functionalities
that empower the development of sophisticated applications. These capabilities include image
and video analysis, feature extraction, object detection, and facial recognition, among others.
Leveraging OpenCV, developers can harness the power of computer vision to enable machines
to interpret and understand visual information, thereby bridging the gap between humans and
technology.

The ultimate goal of the envisioned machine learning application is to facilitate a seamless and
intuitive interaction between users and machines. By incorporating OpenCV's image processing
capabilities, the application can detect and interpret visual cues, paving the way for a more
natural and responsive user experience. This is particularly crucial in HCI systems where the
efficiency and effectiveness of interaction rely on the ability of the system to understand and
respond to user inputs.

The two key characteristics, functionality, and usability, are paramount in designing a
successful HCI system. Functionality refers to the array of functions or services that the system
provides to users. In the case of the proposed application, OpenCV's capabilities contribute to
a rich functionality that includes real-time image processing, object tracking, and gesture
recognition. These functions empower users to interact with the system in a meaningful and
dynamic manner.

Usability, on the other hand, encompasses the level and scope at which the system can operate
and perform specific user purposes efficiently. OpenCV's flexibility and versatility contribute
significantly to the usability of the application by providing a comprehensive set of tools for
image manipulation, feature extraction, and pattern recognition. This ensures that the system
can adapt to diverse user needs and scenarios, enhancing overall user satisfaction.

43
7.1 FUTURE WORK

Some examples that are being implemented in the near future are as follows: Video
Processing using Android Phone:
Mobile devices such as smart phones, iPads and tablet pcs are equipped with cameras, the
demand of the image processing applications increased. These applications need to be faster
and consumes lower power because the mobile device is only powered by a battery . The
hardware technology depends on the semiconductor technology instead we can use an
efficient programming language to write an image processing application for the mobile
devices [4]. Robot Control:
Controlling the robot using gestures considered as one of the interesting applications in this
field proposed a system that uses the numbering to count the five fingers for controlling a robot
using hand pose signs. The orders are given to the robot to perform particular task, where each
sign has a specific meaning and represents different function.

44
 CHAPTER 8 – REFERENCES

[1]. Volume 5, Issue 1, January 2015 ISSN: 2277 128X, International Journal of Advanced
Research in Computer Science and Software Engineering : Research Paper --Gesture
Controlled Computer.
[2]. [Link]
[3]. [Link]
[4]. Volume 43, Issue 1, June 2012 ISSN: 2277128X, International Journal of Advanced
Research in Computer Science and Software Engineering : Research Paper --Hand Data Glove:
A Wearable Real-Time Device forHuman-Computer Interaction [5].
International Journal of Artificial Intelligence& Applications (IJAIA), Vol.3, No.4, July
2012,DOI : 10.5121/ijaia.2012.3412 161- HAND GESTURE RECOGNITION: A
LITERATURE REVIEW
[6]. OpenCV for Computer Vision Applications, Proceedings of National Conference onBig
Data and Cloud Computing (NCBDC’15), March 20, 2015
[7]. [Link]
[8]. [Link] [Link]
[9]
[Link]
objects-in-live-video
[10]. Real Time Object Detection and Tracking Using Deep Learning and OpenCV
Proceedings of the International Conference on Inventive Research in Computing Applications
(ICIRCA 2018) IEEE Xplore Compliant Part Number:CFP18N67-ART;
ISBN:978-1-5386-2456-2

[11]. [Link],2017. Online].Available:[Link]

[12]. Saira Beg, M. Fahad Khan and Faisal Baig, "Text Writing in Air," Journal of
Information Display Volume 14, Issue 4, 2013

[13]. Study on Object Detection using Open CV – Python, International Journal of

ComputerApplications (0975 – 8887) Volume 162 – No 8,
March 2017
[14]. Nidhi, “Image Processing and Object Detection”, Dept. of Computer Applications,
NIT, Kurukshetra, Haryana, 1(9): 396-399, 2015

45
[15]. [Link]
[16]. [Link]
[17]. Volume 43, Issue 1, June 2012 ISSN: 2277128X, International Journal of Advanced
Research in Computer Science and Software Engineering : Research Paper --Hand Data Glove:
A Wearable Real-Time Device forHuman-Computer Interaction [5].
International Journal of Artificial Intelligence& Applications (IJAIA), Vol.3, No.4, July
2012,DOI : 10.5121/ijaia.2012.3412 161- HAND GESTURE RECOGNITION: A
LITERATURE REVIEW

46
47