“AI Virtual Keyboard”

A Major Project Report Submitted to

Rajiv Gandhi Proudyogiki Vishwavidyalaya

Towards Partial Fulfillment for the Award of

Bachelor of Engineering in
CSE(Artificial Intelligence & Machine Learning)

Guided By Submitted By
Prof. Sumit Jain Kartik Joshi (0827AL211030)
Kratika Nenwani(0827AL211031)
Shreya Singh (0827AL211060)
Sam Malviya (0827AL211052)

Acropolis Institute Of Technology and Research, Indore

2024-2025
 Declaration
I hereby declared that the work, which is being presented in the project entitled AI
Virtual Keyboard partial fulfillment of the requirement for the award of the degree of
Bachelor of Technology, submitted in the department of CSE (Artificial Intelligence
& Machine Learning) at Acropolis Institute of Technology & Research, Indore is an
authentic record of my own work carried under the supervision of Prof. Sumit Jain. I
have not submitted the matter embodied in this report for award of any other degree.

Kartik Joshi (0827AL211030)

Kratika Nenwani (0827AL211031)

Shreya Singh (0827AL211060)

Sam Malviya (0827AL211052)

Prof. Sumit Jain

Project Guide
 Project Approval Form
I hereby recommend that the project AI Virtual Keyboard prepared under my
supervision by Kartik Joshi (0827AL211030), Kratika Nenwani
(0827AL211031) , Shreya Singh (0827AL211060) , Sam Malviya
(0827AL211052) be accepted in partial fulfillment of the requirement for the degree
of Bachelor of Technology in CSE (Artificial Intelligence & Machine Learning).

Prof. Sumit Jain

Project Guide

Recommendation concurred in 2024-2025

Dr. Mayur Rathi

Prof Sumit Jain

Project Coordinator
 Acropolis Institute of Technology and Research
Deparment of CSE( Artificial Intelligence & Machine Learning)

Certificate

The project work entitled AI Virtual Keyboard submitted by Kartik Joshi

(0827AL211030), Kratika Nenwani (0827AL211031) , Shreya Singh
(0827AL211060) , Sam Malviya (0827AL211052) is approved as partial
fulfillment for the award of the degree of Bachelor of Technology in CSE (Artificial
Intelligence & Machine Learning) by Rajiv Gandhi Proudyogiki Vishwavidyalaya,
Bhopal (M.P.).

Internal Examiner External Examiner

Name:………………. Name: ……………..

Date: …./…/……….. Date: …./…/………..

 Acknowledgement

With boundless love and appreciation, we would like to extend our heartfelt gratitude
and appreciation to the people who helped us to bring this work in reality. We would
like to have some space of acknowledgement for them.

Foremost, we would like to express our sincere gratitude to our supervisor, Prof.
Sumit Jain whose expertise, consistent guidance, ample time spent and consistent
advices that helped us to bring this study into success.

To the project coordinator Dr. Mayur Rathi & Prof. Sumit Jain for their constructive
comments, suggestions, and critiquing even in hardship.

To the honourable Prof. (Dr.) Namrata Tapaswi, Head, Department of CSE (Artificial
Intelligence & Machine Learning) for her favourable responses regarding the study and
providing necessary facility.

To the honourable Dr. S.C. Sharma, Director, for his unending support, advises and
effort to make possible.

Finally, we would like to pay our thanks to faculty members and staff of Department of
CSE (Artificial Intelligence & Machine Learning) for their timely help and support.

We also like to pay thanks to our parents for their eternal love, support and prayers.
Without them it is not possible.

Kartik Joshi (0827AL211030)

Kratika Nenwani (0827AL211031)

Shreya Singh (0827AL211060)

Sam Malviya (0827AL211052)

 Abstract
Our project centers on developing an AI-powered virtual keyboard that offers a
touch-free typing experience by utilizing OpenCV and MediaPipe libraries. This
virtual keyboard, displayed on the screen, enables users to type by moving their hands
to select letters, providing an accessible and hygienic alternative to physical keyboards.
The project aims to explore gesture-based technology, especially for users who may
have difficulty using traditional input methods or in situations where contactless
interaction is preferable.

The keyboard’s functionality is achieved through real-time hand tracking, with

OpenCV processing video input and MediaPipe precisely detecting hand positions. By
aligning specific hand gestures with corresponding keyboard actions on the screen,
users can hover over or point to keys to simulate typing, creating an intuitive and
contactless interface.

During testing, the virtual keyboard consistently tracked hand movements and
accurately mapped them to characters, achieving strong responsiveness and precision.
Users could type words effectively, demonstrating the system’s practicality for
real-world applications with minimal setup required.

This project highlights the significance of combining AI with gesture recognition to

create assistive technologies, emphasizing how touch-free interfaces can enhance
accessibility and adaptability in typing solutions. Our AI virtual keyboard lays the
foundation for future advancements in gesture-based interfaces, encouraging continued
exploration into AI-driven solutions for more dynamic human-computer interactions.
Keywords:
AI-powered virtual keyboard

Gesture recognition

Computer vision technology

Machine learning

Hand-tracking

Accessibility features

Touchless input

Human-computer interaction
 Table of Content

Declaration i

Project Approval Form ii

Acknowledgement iii

Abstract iv

Table of Content v

List of Figures vii

List of Tables viii

Abbreviations ix

Chapter 1: Introduction

1.1 Rationale 1

1.2 Existing System 1

1.3 Problem Formulation 2

1.4 Proposed System 3

1.5 Objectives 4

1.6 Contribution of the Project 5

1.6.1 Market Potential 5

1.6.2 Innovativeness 6

1.6.3 Usefulness 7

1.7 Report Organization 8

Chapter 2: Review of Literature

2.1 Preliminary Investigation 9

2.1.1 Current System 10

 2.2 Limitations of Current System 11

2.3 Chapter Summary 12

Chapter 3: Requirement Engineering

3.1 Feasiblity Study (Technical, Economical, Operational) 13

3.2 Requirement Collection 15

3.2.1 Discussion 16

3.2.2 Requirement Analysis 16

3.3 Requirements 17

3.3.1 Functional Requirements 17

[Link] Statement of Functionality 18

3.3.2 Nonfunctional Requirments 19

[Link] Statement of Functionality 19

3.4 Hardware & Software Requirements 20

3.4.1 Hardware Requirement (Developer & End User) 21

3.4.2 Software Requirement (Developer & End User) 22

3.5 Use-case Diagrams 23

3.5.1 Use-Case Descriptions 24

Chapter 4: Analysis & Conceptual Design & Technical Architecture

4.1 Technical Architecture 26

4.2 Sequence Diagrams 26

4.3 Class Diagrams 27

4.4 DFD 28

4.5 User Interface Design 28

4.6 Data Design 29

 4.6.1 Schema Definitions 30

4.6.2 E-R Diagram 30

5. Conclusion & Future Scope

5.1 Conclusion 31

5.2 Future Scope 32

REFERENCES 34
 List of Figures

Figure 3.5: Use Case Diagram 23

Figure 4.1: Technical Architecture Diagram 26

Figure 4.2: Sequence Diagrams 26

Figure 4.3: Class Diagrams 27

Figure 4.4: DFD 28

Figure 4.5: User Interface Design 28

Figure 4.6: Data Design 29

Figure 4.6.2: E-R Diagram 30

 List of Tables

Table 3.1:Requirement Collection 15

Table 3.2:Use Case Function 25

 Abbreviations

Abbr1: AI - Artificial Intelligence

Abbr2: AR – Augmented Reality

Abbr3: VR – Virtual Reality

Abbr4: CNN – Convolutional Neural Network

Abbr5: RNN – Recurrent Neural Network

 CHAPTER 1: INTRODUCTION

1.1 Rationale
The rationale behind the "AI Virtual Keyboard Using Computer Vision" project lies in
the increasing demand for more advanced and contactless human-computer interaction
methods. Traditional input systems such as physical keyboards present several
challenges related to hygiene, wear and tear, and accessibility. By employing computer
vision, this project introduces a feasible solution that allows users to interact with a
computer through simple hand gestures, making the interaction touch-free and more
intuitive. The project addresses needs for non-contact technology, which has gained
significance in light of health concerns and the push towards innovative user interfaces.

1.2 Existing System

Existing systems for text input, including physical keyboards, touchscreen interfaces,
and gesture recognition technologies, have notable limitations. Physical keyboards are
widely used but come with drawbacks such as requiring physical contact, which raises
hygiene concerns among people and leads to wear and tear, requiring regular
maintenance to remain functional. Touchscreen keyboards, though providing a digital
and portable alternative, present issues related to screen size constraints and latency
issues , resulting in a higher error rate and user fatigue, especially for those more
comfortable with physical keys.

Gesture recognition systems, on the other hand, utilize specialized cameras to interpret
hand movements for input. While these systems eliminate the need for contact, they
tend to be complex and expensive, requiring sophisticated hardware setups that can be
sensitive to changes in lighting and background, impacting their accuracy and usability.
Additionally, voice input technologies offer hands-free interaction but prove
impractical in noisy environments or places where speaking aloud might compromise
privacy or cause disturbance.
Overall, these limitations reveal a gap in the market for a solution that is contact-free,
cost-effective, reliable in different settings, and accessible to users of all technical skill
levels. The challenges of maintaining hygiene, minimizing error rates, and reducing
the need for expensive hardware point to the need for an innovative, adaptable
approach to text input.

1.3 Problem Formulation

The primary problem addressed by this project is the limited accessibility and usability
of traditional input methods for individuals , particularly those who are unable to use a
standard keyboard or touchscreen due lack of availability of such systems. While
various assistive technologies have been developed, none provide a universally
accessible solution that allows users to interact with devices efficiently without
physical contact. Physical keyboards and touchscreens, though commonly used, are not
feasible for individuals concerned with hygiene issues , as they require tactile
interaction. Additionally, existing virtual keyboards, which operate on touchscreens or
through onscreen typing, still rely on manual input, making them ineffective for users
who avoid physical contacts.[1]

Current gesture-based input methods, such as those using motion sensors or cameras
also face significant limitations in terms of accuracy and practical usability. These
systems often misinterpret gestures or lack the precision required for tasks like typing.
Most gesture recognition systems focus on broad, large movements, which are
unsuitable for typing as they do not detect the fine motions of hand movement needed
for virtual keyboard input. The lack of an intuitive, reliable solution for hands-free
typing leaves a significant gap in accessible technology. Therefore, the problem to be
solved by this project is the creation of a gesture-controlled virtual keyboard using
computer vision and AI, which recognizes precise hand and finger movements and
converts them into accurate text. This system aims to provide an accessible, fast, and
user-friendly typing solution for users with physical limitations

.
1.4Proposed System
The AI-powered virtual keyboard aims to provide an efficient, hands-free typing
experience using gesture recognition through computer vision. This system eliminates
the need for a physical keyboard by recognizing hand gestures and translating them
into text input. Implementing a virtual keyboard in applications can greatly enhance
accessibility for users , ensuring that the application meets certain predefined rules of
libraries used and standards [1]. The core of the system is designed to be accessible,
especially for individuals , offering a convenient and innovative alternative to
traditional typing method.

● Gesture Recognition and Computer Vision: The proposed system uses

computer vision to track and interpret hand gestures in real time. A camera
captures the user's hand movements, which are then processed using computer
vision algorithms to detect key hand landmarks such as fingers, palms, and
joints. Libraries like MediaPipe can be used to detect these landmarks, and deep
learning models like Convolutional Neural Networks (CNNs) help recognize
the hand movements accurately.
● Machine Learning for Gesture Classification: To improve the accuracy of the
gesture recognition, the system uses machine learning algorithms. The system
learns from the user’s interactions, adapting over time to improve its ability to
recognize specific hand movements. Supervised learning can be employed to
train the system on various hand gestures and their corresponding actions.

● Virtual Keyboard Layout and Interaction: The system presents a virtual

keyboard on the screen, with a layout similar to traditional keyboards, such as
QWERTY, or customized layouts depending on the user’s needs.

● Multi-Platform Compatibility: The proposed virtual keyboard system is

designed to work across various platforms. It can be integrated into desktop
computers, mobile devices, making it versatile and adaptable. The system is
compatible with common operating systems such as Windows, macOS, Linux,
and mobile OS platforms like Android and iOS.
 ● Real-Time Processing and Performance Optimization: A key challenge in
developing a gesture-based system is ensuring real-time performance. The
system processes the camera feed and recognizes gestures without noticeable
delay. To achieve this, the system uses efficient algorithms that optimize latency
and computational power, allowing for rapid inputs.

● Customization and Personalization: The system allows users to customize

various aspects of the virtual keyboard. They can adjust the size of the keys, the
sensitivity of the gestures, and even the typing speed to fit their needs. For users
desiring a comparatively slower input , the system can be set to recognize
larger, slower gestures, while those with greater dexterity can use smaller,
quicker movements.

● Accessibility Focus: One of the main goals of the system is to enhance

accessibility. The system provides an alternative typing method that enables
users to interact with their devices using only their gestures. Moreover, the
system is designed to be user-friendly for people with limited technical
knowledge, with easy setup and intuitive controls.

1.5Objectives
The objective of the proposed AI-powered virtual keyboard system is to provide an
innovative, hands-free typing solution that utilizes gesture recognition and computer
vision technology. The system aims to enable users to input text through intuitive hand
movements, eliminating the need for a traditional physical keyboard. By capturing and
analyzing hand gestures in real-time, the system translates these gestures into text,
making it a valuable tool for individuals seeking an alternative typing method. The main
goal is to create a typing system that empowers users to interact with devices using only
their gestures, improving accessibility for a wider range of people.[2]

Furthermore, the proposed system aims to support multiple platforms, including

desktops, smartphones, ensuring its versatility across different devices. Overall, the
system's objectives are to improve the typing experience by making it more accessible,
flexible, and adaptive, thus offering a new and inclusive way for users to engage with
technology.

1.6Contribution of the Project

1.6.1 Market Potential

The market potential for the proposed AI-powered virtual keyboard system is
significant, given the increasing demand for assistive technologies, hands-free
solutions, and personalized user experiences. The key factors contributing to its
market potential are:

Growing Demand for Accessibility Solutions:

There is a rising global demand for assistive technologies that can help individuals
having limited mobility, to interact with devices. The system provides a hands-free
typing solution that can be used by individuals even without having a physical input
keyboard available in hand. This makes it highly relevant in the growing assistive
technology market.

Increasing Use of Virtual Reality (VR) and Augmented Reality (AR):

The rapid growth of VR and AR technologies opens up new use cases for the
virtual keyboard system. In VR/AR environments, traditional input devices like
keyboards and mice are often impractical, making hands-free typing a valuable
tool. The proposed system can be integrated into VR/AR applications, where
gesture-based input is essential, thus expanding its potential customer base to
include users in gaming, education, design, and entertainment industries.

Rising Adoption of Gesture-Based Technology:

With the increasing popularity of devices like smartphones, tablets, and

smartwatches that use gesture recognition technology, the market is already
familiar with similar concepts. There is an emerging trend toward hands-free input
methods that allow users to interact with devices using gestures. The proposed
system, which uses machine learning and computer vision for accurate gesture
recognition, positions itself as an advanced solution within this trend.

Shift Toward Personalized User Experiences:

As consumers demand more customizable and adaptive technologies, the

AI-powered virtual keyboard system's ability to personalize the typing experience
is a strong market driver. Users can adjust keyboard layouts, gesture sensitivity,
and typing speed, making the system highly appealing for individuals who need
unique settings or personal preferences.

Global Reach:

The system’s efficiency allows users from various backgrounds to use it

comfortably, which expands its market potential to global markets. As businesses
and organizations increasingly cater to international audiences, the need for a
system to type without using a physical keyboard increases exponentially, creating
a channel for commercialization of the system.

Healthcare and Rehabilitation Market :

In the healthcare sector, especially in rehabilitation settings, where the hygiene
concerns need to be taken care the most, the virtual keyboard can be a vital tool for
patients or the staff to provide necessary information that too without any physical
contact .

1.6.2 Innovativeness
The innovativeness of the proposed AI-powered virtual keyboard system reflects
from its use of cutting-edge technologies like gesture recognition, computer vision,
and machine learning to enable hands-free typing. The system tracks users' hand
movements in real-time through computer vision algorithms, which interpret
gestures and translate them into text input. The virtual keyboard has evolved from
simple alphanumeric input to more advanced features such as predictive text,
gesture controls,changing the way users interact with their devices[2].

Unlike traditional input methods that rely on physical interaction, this system uses
advanced gesture recognition to offer a seamless, intuitive typing experience. Over
time, the system’s machine learning algorithms adapt to the user’s unique gestures,
enhancing accuracy and making the typing experience increasingly efficient. This
adaptive capability allows for personalized interactions, making the system
intuitive for users with varying levels of dexterity.

Furthermore, the system's customizable virtual keyboard layouts set it apart from
other solutions. The ability to personalize the keyboard layout, gesture sensitivity,
and typing speed ensures a more user-centered experience, catering to individual
needs and preferences, especially for those with specific accessibility
requirements. Its cross-platform compatibility (desktop and mobile environments)
further enhances its innovation, making it versatile across a range of devices and
use cases. Whether in a smartphone, or a traditional desktop, the system offers a
hands-free solution that can be used universally. This innovative blend of adaptive
gesture recognition and versatile, inclusive design positions the system as a
groundbreaking tool in both assistive technology and user experience design.

1.6.3 Usefulness
The usefulness of the proposed AI-powered virtual keyboard system lies in its
ability to address a wide range of practical needs, particularly in the fields of
accessibility, convenience, and efficiency. For individuals facing mobility
challenges, this system provides a hands-free typing solution that eliminates the
need for traditional keyboards or touchscreens. Users can interact with their
devices through simple hand gestures, making it particularly valuable for
individuals. This accessibility feature not only promotes independence but also
ensures that users can continue to engage with technology in a meaningful way,
improving their overall efficiency.

In addition to its accessibility benefits, the system offers significant convenience

and efficiency for a broad range of users. It can be used in environments where
 traditional input methods may be impractical, such as in clean rooms, hospitals, or
laboratories, where hygiene is critical and hands-free operation is necessary. The
system's adaptability allows it to be seamlessly integrated into VR/AR
environments, where users can type or interact with virtual objects without needing
a physical keyboard.

1.7 Report Organization

The report is organized to provide a comprehensive and logical flow of information,

guiding the reader through the entire process of the AI-powered virtual keyboard
system's development.

Chapter 1: Introduction sets the stage by outlining the project’s rationale, objectives,
contributions, and potential impact, offering an overview of the system’s purpose and
benefits.

Chapter 2: Literature Review examines existing technologies, solutions, and research in

the fields of gesture recognition, virtual keyboards, and accessibility, providing context
for the new approach.

Chapter 3: System Design and Architecture focuses on the technical aspects of the
system, explaining its design, architecture, and the integration of key technologies like
gesture recognition, computer vision, and machine learning.

Chapter 4: Analysis and Conceptual Design of Technical Architecture, the focus will be
on a detailed exploration of the technical aspects of your system.

Chapter 5: Conclusion and Future Work wraps up the findings, summarizing the
achievements and suggesting areas for future development, such as improving gesture
recognition and expanding platform compatibility.

The References section lists all sources cited throughout the report, while the
Appendices provide additional material, such as user manuals, for deeper technical
insights.
 CHAPTER 2: REVIEW OF LITERATURE

2.1 Preliminary Investigation :

The preliminary investigation focused on exploring the existing knowledge and
technologies related to gesture recognition and virtual keyboards. This stage was
crucial for understanding the current advancements in human-computer interaction and
identifying areas for improvement. We began by reviewing academic papers, industry
reports, and technical documents to see how gesture-based input methods have
developed, evolving from basic touch interfaces to more advanced, contactless
solutions. This research included studying how computer vision and machine learning
have improved the accuracy and responsiveness of gesture recognition systems.

A key part of our investigation looked at the progression of virtual keyboard systems.
Early versions of these systems were limited to physical or on-screen keyboards,
requiring direct contact and offering little flexibility for users with diverse needs. As
gesture recognition technology emerged, virtual keyboards started to incorporate
touchless interaction, though early systems often struggled with issues related to
accuracy, speed, and adaptability to different environments or user capabilities. By
examining these early attempts, we gained insight into foundational work that paved
the way for more advanced systems and learned about methods used to interpret hand
movements in real-time.[3]

We also explored how gesture-based technology is applied in various platforms,

particularly focusing on mobile and desktop applications. Our research highlighted
various tools and technologies, such as computer vision techniques, hand-tracking
algorithms, and machine learning models, that enable effective gesture recognition. By
examining these examples, we identified both successful applications and common
challenges, helping us determine which design principles and technologies could be
valuable for our AI-powered virtual keyboard system. This initial research provided a
solid foundation for developing a more accurate and user-friendly virtual keyboard,
aligned with our project goals.
2.1.1 Current System :
The Current System section examines the technologies and solutions currently used for
virtual keyboards and gesture-based input methods. Virtual keyboards have come a
long way from their early versions, now often incorporating gesture recognition,
machine learning, and computer vision to allow users to type without a physical
keyboard. Today’s virtual keyboards are commonly found on platforms like mobile
devices and desktop computers, with each approach offering varying levels of
functionality and adaptability.

Most current virtual keyboards still rely on touch-based interactions, like on-screen
keyboards on smartphones or tablets, where direct contact is required for typing.
Recently, gesture recognition has been introduced to allow for touchless typing,
although it is still limited in accuracy and often requires specialized hardware like
infrared sensors or depth-sensing cameras. In some advanced systems, hand-tracking
technology using computer vision enables users to interact with virtual keyboards
through gestures, although these systems may struggle with accurate gesture detection
in different lighting conditions or for users with varied physical needs.[4]

Customization and accessibility in existing systems also vary. While some allow basic
adjustments, like resizing keys or changing layouts, very few are designed to adapt to
users with different physical abilities or specific preferences. Additionally, many
current systems lack comprehensive support for multiple languages, limiting their
usability for a diverse range of users.

Overall, while current virtual keyboard systems demonstrate promising developments,

they still face challenges with accuracy, adaptability, and accessibility, highlighting
areas for improvement in our own AI-powered virtual keyboard project. By addressing
these limitations, our project aims to provide a more intuitive and accessible solution
for users seeking a reliable contactless typing experience.
 2.2 Limitations of Current System :

The limitations of current virtual keyboards and gesture-based input systems highlight the need
for more advanced solutions. While these systems have made progress in enabling touchless
interactions, they still fall short in accuracy, accessibility, and adaptability for diverse users.

● Inconsistent Gesture Recognition: Many systems struggle to interpret gestures

accurately, especially in different lighting conditions or when users have unique
gestures. This inconsistency can cause errors and delays, making the experience
frustrating for users.
● Limited Accessibility: Although some systems aim to be accessible, they often don’t
fully address the needs of users with severe mobility impairments or specific physical
requirements, limiting their usability for many people.
● Lack of Multilingual Support: Many virtual keyboards lack strong support for
multiple languages, making them less practical for non-English speakers or users who
need to switch between languages.
● Minimal Customization Options: While some systems offer basic customization,
such as adjusting keyboard layout or key size, they don’t typically provide in-depth
personalization options to suit individual user preferences.
● Poor Cross-Platform Compatibility: Many virtual keyboards work well on specific
devices, like smartphones or tablets, but often face challenges on other platforms, such
as desktop computers.
● Dependence on Specialized Hardware: Some advanced gesture systems require
specialized hardware, like depth-sensing cameras or infrared sensors, which can be
costly and limit availability for the average user.

These limitations underscore the need for a virtual keyboard solution that offers higher accuracy,
better accessibility, and wider compatibility, which our AI-powered virtual keyboard aims to
address. By focusing on these areas, our project seeks to create a more reliable, user-friendly typing
experience that can adapt to various user needs and environments.[7]
2.3 Chapter Summary :

This chapter summary concludes the Review of Literature by gathering key insights from our
research into current gesture recognition and virtual keyboard systems. It outlines the main points
from the preliminary investigation, highlighting the evolution of technologies in this field and
examining existing systems. We looked at both the strengths and limitations of these systems,
noting issues such as inconsistent gesture accuracy, limited accessibility features, and challenges
with cross-platform compatibility.

These findings emphasize the need for an improved solution, setting the stage for our proposed
AI-powered virtual keyboard system. Our project aims to address these challenges by offering
more precise gesture recognition, enhanced accessibility, and broader compatibility across
different devices. This summary leads into the next sections of the report, which will focus on the
design and structure of our new system and how it aims to overcome the current limitations.[8]
 CHAPTER 3: REQUIREMENT ENGINEERING

3.1 Feasibility Study (Technical, Economical, Operational)

Technical:

Technical feasibility focuses on whether the technology and resources required to build the AI
virtual keyboard are accessible and sufficient. This project relies heavily on computer vision,
specifically tracking hand movements to simulate keyboard input. To achieve this, two main
libraries—OpenCV and MediaPipe—will be used.

OpenCV, a popular open-source library for image processing, helps capture and analyze the video
stream from a camera. MediaPipe, another advanced library, provides efficient tools for detecting
and tracking hands and their positions, which are essential for recognizing finger movements that
correspond to keyboard keys. These tools enable real-time tracking with minimal delay, a critical
requirement for a smooth and responsive user experience.

In terms of hardware, the system needs a camera to capture hand movements and a computer with
moderate processing power. The project will need proficiency in Python for all tasks, optimizing
the code or switching to more efficient implementations might be needed if performance becomes
an issue.[6]

Overall, this project is technically feasible because the required libraries are available and
relatively straightforward to implement, provided the device meets certain specifications. If
performance requirements increase, there are options to improve efficiency through code
optimization or hardware upgrades.

Operational:

Operational feasibility evaluates how effectively the AI virtual keyboard would work for its
intended users and environments. In this case, the goal is to create a keyboard interface that users
can control through hand gestures rather than physical touch.

For the project to be operationally feasible, the system must be accurate, user-friendly, and
responsive. The AI keyboard should have a clear layout, with each key easily identifiable on the
screen. As users gesture to press a virtual key, the system should provide immediate visual
feedback, such as highlighting the pressed key, so users know their input was registered correctly.
This feedback is critical in making the system feel responsive and easy to use.

Potential operational challenges include user accuracy in different lighting conditions, and the
need for the keyboard to adapt to varying hand sizes and positions. For example, bright lighting
might improve accuracy by reducing shadows, while poor lighting could make it harder for the
system to detect hands accurately. To address this, adaptive brightness controls can be
implemented to make the system work in various environments.

By designing the interface with usability considerations in mind, such as requiring minimal hand
movement to interact with the keyboard, the project can enhance the user experience and
minimize potential strain or fatigue. Ultimately, the AI virtual keyboard is operationally feasible
as long as it’s designed to be responsive, visually clear, and adaptable to different environments
and user needs.

Economical:

Economic feasibility examines the costs involved in developing and deploying the AI virtual
keyboard. Because this project uses open-source libraries (OpenCV and MediaPipe), there are no
extra fees, which significantly reduces costs. For a personal or academic project, the economic
requirements are minimal, as there are no additional financial burdens beyond the initial hardware
investment and development time.

For developers or researchers, the main economic factors are time and labor. The primary cost
comes from the time invested in software development, debugging, and testing.[7] If this project
were to be scaled for commercial use, other expenses could include hiring more developers,
implementing regular updates, and possibly covering maintenance costs.

If the project were to be implemented commercially, additional financial considerations might

include marketing, user support, and ongoing maintenance for performance updates. However, for
an academic or research setting, the AI virtual keyboard is economically feasible because it only
requires basic development resources and hardware already available to most developers.
3.2 Requirement Collection

For our AI virtual keyboard project, we identified and implemented a comprehensive set of
features, resources, and user-centered expectations to ensure successful development and
deployment. Our functional requirements included real-time hand tracking using a camera to
capture finger movements and gestures, allowing users to “press” virtual keys. We created a
visually accessible keyboard interface, adding intuitive feedback, such as key highlighting, to
facilitate clear user interaction.

We addressed non-functional requirements by optimizing performance, accuracy, and device

compatibility. The system was designed to detect gestures quickly, operate in varied lighting
conditions, and achieve a minimum accuracy rate of 85% for reliable typing. System requirements
were also met, including a camera with 720p resolution and a computer with moderate processing
power, alongside OpenCV and MediaPipe libraries for gesture recognition in Python.

User requirements focused on ease of setup, minimal physical effort, and customizable options,
such as calibration and feedback, to enhance accessibility for users with limited mobility. To
address privacy and security, we ensured local processing to avoid storing or transmitting video
data externally. This approach has made our virtual keyboard efficient, user-friendly, and
adaptable across various devices and environments.

Table 1.1 Requirement Collection

 3.2.1 Discussion
The AI virtual keyboard project presents an innovative solution for by utilizing real-time hand
tracking and gesture recognition. This project addresses a need for alternative input methods,
for those looking to minimize physical interaction with devices. Through the integration of
OpenCV and MediaPipe, the system detects hand landmarks and gestures, allowing users to
interact with an on-screen keyboard. This method provides a unique challenge in achieving a
balance between accuracy and responsiveness to create a smooth user experience.

Key areas of discussion include the technical complexity of maintaining real-time performance
across different devices and lighting conditions.[6] Effective gesture recognition requires
accurate detection despite environmental variations, which introduces potential challenges in
managing system consistency and minimizing latency. Additionally, ensuring that the virtual
keyboard is intuitive and easy to use is critical, requiring a user-centric design.

Privacy considerations are also significant, as the use of a live camera feed brings potential
data security concerns. Emphasizing local data processing and minimal data storage helps
safeguard user privacy, making the system more trustworthy. Lastly, while the project is
designed to be economically feasible by relying on open-source libraries, future commercial
deployment could require further investments in optimization and user support. Ultimately,
this project combines technical innovation with practical accessibility solutions, offering both
a unique interface and an inclusive approach to virtual typing.

3.2.2 Requirement Analysis

The requirement analysis for our AI virtual keyboard project involved carefully examining
and refining the gathered requirements to ensure they were achievable, relevant, and aligned
with the project's goals. We assessed the functional requirements—such as real-time hand
tracking, gesture recognition, and a responsive keyboard layout—by evaluating the technical
feasibility of achieving smooth performance. The project relies on MediaPipe for hand
tracking and OpenCV for processing video streams, so understanding the capabilities and
limitations of these libraries was critical in optimizing detection accuracy and responsiveness.

Non-functional requirements were equally prioritized, focusing on the overall quality of the
 user experience, including performance, reliability, and accessibility. The system needed to
detect gestures quickly for a seamless typing experience, with high accuracy to minimize
input errors—especially under varied lighting conditions. Compatibility with devices of
moderate specifications was essential, as hardware differences could impact performance and
user experience, making this analysis a key part of the project.[8]

We also thoroughly analyzed user requirements such as ease of setup and customization
options to ensure they aligned with accessibility goals. Additionally, privacy considerations
were a significant focus, with measures in place to prevent unintended data storage or
transmission by processing all data locally on the device, which helped establish user trust.
This detailed analysis ensured that each requirement was validated for feasibility, prioritized
according to user impact, and refined to make the final system both technically robust and
user-centered. The result was a well-rounded, functional AI virtual keyboard.

3.3 Requirements

3.3.1 Functional Requirements

The functional requirements for the AI virtual keyboard define the core actions and
capabilities that the system must perform to achieve its primary objective of hands-free,
gesture-based typing. These requirements focus on specific functionalities that will allow
users to interact with the keyboard using only hand movements:

● Real-Time Hand Tracking: The system must capture live video from a camera and
use MediaPipe’s hand detection model to recognize hand landmarks (like fingertips and
joints) in real-time. This enables precise detection of hand positions and finger
movements, essential for simulating key presses.
● Gesture Recognition for Key Press Simulation: The system should interpret specific
hand gestures, such as pointing or hovering over a key, as input signals to simulate
pressing that key on the virtual keyboard[3]. This requires mapping hand movements to
corresponding keys on the interface, allowing users to type accurately using gestures
alone.
● Interactive Virtual Keyboard Display: A visual keyboard layout with all standard
 keys (letters, numbers, symbols) must be displayed on the screen. This virtual keyboard
provides an intuitive interface, allowing users to identify and “press” keys by
positioning their hand over the appropriate areas.
● Visual and Real-Time Feedback: Each key press should trigger immediate feedback,
such as highlighting the selected key, to confirm the user’s input. This feedback loop is
vital for accuracy and helps users know when their gesture has been successfully
registered.
● Calibration and Customization Options: The system should offer a calibration
feature, allowing users to adjust settings such as sensitivity, hand size, and screen
distance to optimize recognition. Additionally, users should be able to customize
certain features, like feedback options, for a more tailored experience.
● Error Handling and Input Validation: The software should handle common input
errors, such as unintentional gestures, by including mechanisms to filter out accidental
touches or gestures. This ensures accurate typing and enhances the overall usability of
the keyboard.

[Link] Statement of Functionality

The AI virtual keyboard is designed to enable users to interact with a digital keyboard
through hand gestures. Using real-time hand tracking via a camera, the system detects and
interprets hand and finger movements, allowing users to simulate key presses without
physically touching a keyboard. Key features include an on-screen visual keyboard
display, which users can interact with by positioning their fingers over virtual keys. Each
key press is confirmed with visual feedback, such as highlighting the selected key,
ensuring users receive immediate confirmation of their input. Additionally, the system
offers customization options, such as sensitivity adjustments, calibration for hand size and
screen distance, and feedback settings, allowing users to personalize the keyboard to suit
their specific needs and environment. By combining gesture recognition, real-time
feedback, and customization, the AI virtual keyboard aims to deliver an accessible,
intuitive typing experience that meets the demands of diverse user groups.

3.3.2 Nonfunctional Requirements

Non-functional requirements define the quality attributes and overall characteristics of the AI
virtual keyboard, ensuring that the system not only meets its functional goals but also
provides a high-quality user experience. These requirements focus on performance, usability,
reliability, and accessibility.

First, performance is a critical non-functional requirement. The system must process hand
movements in real-time, with minimal latency, to ensure a smooth and responsive typing
experience. Delays in gesture recognition can lead to frustration and hinder usability, so the
system should be optimized to detect and process hand movements with a response time of
under a few milliseconds.

Accuracy and precision are also vital. The system should recognize hand gestures with at
least 90% accuracy, minimizing false positives and missed key presses. This high level of
precision is crucial for effective typing and user satisfaction, especially when used in various
lighting conditions or with different hand sizes.

Device compatibility is another important aspect. The virtual keyboard must function devices
like laptop, as long as they have a camera and meet minimum processing specifications. The
software should be adaptable to different hardware configurations, ensuring broad
accessibility without requiring high-end equipment.

[Link] Statement of Functionality

The AI virtual keyboard is designed to offer an alternative input method by enabling users
to type using hand gestures rather than physically pressing keys. This is achieved through
the integration of real-time hand tracking and gesture recognition technology. The system
utilizes a camera to capture hand movements, which are processed by the MediaPipe
framework to detect key landmarks on the user's hands, such as the fingertips. When a
user positions their hand over a specific key on the virtual keyboard, the system detects
the gesture and registers it as a key press. The system then simulates the input on a virtual
on-screen keyboard, allowing the user to type without the need for traditional keyboard
interaction.

The system provides immediate visual feedback to the user, such as highlighting or
 changing the color of the selected key when pressed. This ensures that users know their
input has been successfully recognized and improves the overall user experience. In
addition, the keyboard layout can be customized to accommodate different users'
preferences, with options for adjusting sensitivity, hand size calibration, and feedback
settings. This customization ensures the system can adapt to a wide variety of users and
environments, enhancing accessibility.[8]

Further, the virtual keyboard interface is designed to be simple, intuitive, and easy to
navigate. It includes all standard keys (letters, numbers, punctuation), with clear visual
cues for interaction. The goal is to make typing intuitive and efficient while minimizing
the effort required from the user. This allows for hands-free operation, which can be
particularly useful for individuals with limited mobility or in scenarios where traditional
typing is impractical. Ultimately, the AI virtual keyboard aims to provide an inclusive,
accessible, and ergonomic typing experience through gesture-based control, while
maintaining responsiveness, accuracy, and ease of use.

3.4 Hardware & Software Requirements

3.4.1 Hardware Requirement (Developer & End User)

For developers, the hardware requirements include a computer with a minimum Intel i5
processor, 4 GB of RAM, and a camera with at least 720p resolution for accurate hand
tracking during development and testing. A basic GPU can help improve performance,
especially for real-time gesture processing, but it’s not mandatory. Developers will also need
a suitable operating system (Windows, macOS, or Linux) and the necessary software tools
like Python, OpenCV, and MediaPipe for building and optimizing the virtual keyboard.

For end users, the hardware requirements are more general but still need to support the
system’s functionality. Users should have a device with a camera that has a resolution of at
least 720p, enabling accurate hand gesture detection. The system will work on devices with
an Intel i3 processor (or equivalent) and 4 GB of RAM, ensuring smooth performance for
real-time input recognition.[7] The virtual keyboard should be compatible with desktops,
 laptops, and some tablets or mobile devices with front-facing cameras, providing broad
accessibility across various hardware configurations. Overall, both developers and end users
need devices capable of running the software efficiently and supporting real-time processing
of video and gestures.

3.4.2 Software Requirement (Developer & End User)

For developers, the software requirements include a development environment capable of
running Python, as the AI virtual keyboard is built using this language. The essential libraries
are OpenCV for image processing and MediaPipe for hand gesture detection and tracking. A
suitable Integrated Development Environment (IDE) like PyCharm or VS Code is
recommended for coding and debugging. Developers will also need to install additional
dependencies for system optimization, such as NumPy and TensorFlow, depending on the
complexity of the gesture recognition model. Additionally, version control software like Git
and a platform like GitHub for collaboration and code management is recommended.

For end users, the primary software requirement is having an operating system that supports
the application, such as Windows, macOS, or Linux. The system requires a compatible
browser (if the application is web-based) or a standalone software application to run the
virtual keyboard. The software must be lightweight to ensure smooth performance, even on
lower-end hardware. Users will also need the Python Runtime Environment if running the
software locally, though in most cases, a packaged executable or web-based interface would
eliminate this need. Additionally, a secure virtual keyboard design must focus on preventing
keylogging attacks and ensuring that input is encrypted to protect users from security
breaches[3].

3.5 Use-Case Diagram

 Fig 3.5 Use-case diagram

3.5.1 Use-Case Description

The Use Case Diagram illustrates the key interactions between the User and the AI
Virtual Keyboard System built with OpenCV and MediaPipe libraries. The main
goal of this system is to provide a hands-free, gesture-based typing solution. The
following use cases highlight each interaction:

● Open Virtual Keyboard: The user starts the virtual keyboard application,
initiating the hand detection process.
● Hand Detection: The system uses the camera to detect the user’s hand, a
function enabled by MediaPipe’s hand-tracking features.
● Gesture Recognition: The AI system identifies specific hand gestures, such as
tapping or hovering, to correspond to virtual key selections on the keyboard.
● Key Press Detection: Based on the gestures and hand positions, the system
detects the intended key press by the user.
● Text Input: The selected keys are processed as text input and appear in a
designated text area in real time.
● Display Suggestions: Based on the input, the system can display word
suggestions to assist the user in completing words faster.
● Close Virtual Keyboard: The user can end the session by closing the virtual
keyboard application.

Each use case is contained within the system boundary labeled "AI Virtual
Keyboard System," indicating the scope of this application's functionality and
interactions. The User interacts with the system to type text without physically
pressing any keys, allowing for a novel hands-free typing experience.
Table 3.2 Use Case Function
 CHAPTER 4: ANALYSIS & CONCEPTUAL DESIGN &
TECHNICAL ARCHITECTURE

4.1 Technical Architecture

Fig 4.1 Technical Architecture Diagram

4.2 Sequence Diagrams

Fig 4.2 Sequence Diagram

4.3 Class Diagrams

Fig 4.3 Class Diagram

4.4 DFD

Fig 4.4 DFD

4.5 User Interface Design

Fig 4.5 User Interface Design

4.6 Data Design

Fig 3.6 Data design Diagram

4.6.1 Schema Definitions
4.6.2 E-R Diagram

Fig 4.6.2 E-R Diagram

 CHAPTER 5: CONCLUSION AND FUTURE SCOPE

5.1 CONCLUSION

The AI Virtual Keyboard project aims to create a more accessible and intuitive way to
interact with digital devices. Using OpenCV for image processing and MediaPipe for
hand gesture detection, the system enables users to type without touching a physical
keyboard. Instead, users type by making hand gestures over virtual keys on the screen,
which the system detects and processes in real-time. This hands-free typing method
offers an innovative alternative for individuals with disabilities or anyone looking for a
non-traditional typing solution.

A key strength of this virtual keyboard is its adaptability. The system allows users to
adjust settings like hand size, sensitivity, and screen distance, ensuring a comfortable
experience across different needs and environments. Whether a user requires high
sensitivity or prefers a more responsive interface, these customization options make the
system accessible to a broad range of users.[4]

Throughout development, we addressed technical challenges such as ensuring real-time

performance, achieving accurate gesture recognition, and making the software
compatible with different devices. The system processes hand movements with minimal
delay for a smooth typing experience. Despite obstacles like varying lighting conditions
or hardware differences, the system is designed to remain accurate and reliable.

To further enhance user experience, the virtual keyboard includes real-time feedback
features, such as highlighting the selected key when a gesture is detected. Additionally,
error-handling features guide users by providing feedback in case of misdetection or
input issues, helping users adjust gestures or settings as needed. This keeps the system
user-friendly and reliable, even in challenging environments.
Overall, the AI Virtual Keyboard has the potential to redefine digital interactions. It
provides an inclusive, ergonomic typing experience that is suitable for a variety of users,
from those with physical limitations to those looking for an innovative typing method.
With its focus on user-centered design and cutting-edge gesture recognition, the AI
Virtual Keyboard offers a promising alternative to traditional keyboards, making
technology more accessible and future-ready.

5.2 FUTURE SCOPE

The future scope of the AI Virtual Keyboard project is promising, with many potential
advancements in gesture-based interaction and accessibility. As computer vision,
machine learning, and AI technology progress, our system could be improved to achieve
even greater accuracy and responsiveness. One area for enhancement is integrating
advanced machine learning algorithms, allowing the system to adapt to each user’s
unique hand gestures over time, which would further increase precision and reduce error
rates.

Expanding compatibility across a wider range of devices is another exciting direction.

By adapting the virtual keyboard for use on smart glasses, augmented reality (AR)
headsets, and even wearables like smartwatches, users could interact with the keyboard
on the go or in mobile settings. For instance, users could type in an AR environment
simply by gesturing in the air, enabling a more immersive and hands-free experience.

Adding multilingual support is also an important area for growth. By integrating

language-specific keyboards and gesture recognition for different scripts, the system
could be accessible to a global audience. Exploring voice input integration could allow
seamless switching between gesture and voice commands, making the system even more
versatile and accessible for people with various needs.

There is also potential for integration with other technologies, like IoT devices, smart
home systems, and gaming platforms, where gesture-based controls could become a
primary method of interaction. Features such as predictive text, personalized
suggestions, and custom gesture commands could make the virtual keyboard even more
user-friendly and efficient. As gesture recognition technology advances, the AI Virtual
Keyboard could evolve into a comprehensive, multi-modal input solution, offering a
highly interactive and hands-free experience applicable across various fields, from
healthcare and education to entertainment and workplace settings.
 REFERENCES

[1] Devox Software. (2023, April 10). Virtual Keyboard Implementation: Enhancing
Accessibility for Web Applications. Available:

[2] Udemy. (2023, February 28). Ultimate Guide to Social Media Web Development &
Integration

[3] Johnson, L. (2023, February 20). Designing a Secure Virtual Keyboard for Online
Banking.

[4] Kumar, R. (2023, June 18). The Impact of Virtual Keyboards on User Experience in
Touchscreen Devices.

[5] X. Zhang and Y. Li, "Hand Gesture Recognition for Virtual Input Devices Using
Machine Learning," *Pattern Recognition Letters*, vol. 162, pp. 45-52, 2023.

[6] VirtualTech Solutions. (2023, August 30). Best Practices for Implementing a Virtual
Keyboard on Web Platforms.

[7] T. Brown and M. Green, "Enhancing User Experience with Virtual Keyboards: A
Comprehensive Review," *Journal of User Interface Engineering*, vol. 26, no. 2, pp.
89-105, 2021.

[8] N. Patel and R. Sharma , "Gesture-Based AI Systems for Contactless Input: A

Review of Virtual Mouse and Keyboard Technologies," Journal of Intelligent Systems
and Applications, vol. 16, no. 2, pp. 78-85, 2023.