PRESENTS
SOLUTION NAME
A103
Voice in Signs
INTERNATIONAL WOMEN HACKATHON Guru Nanak Institutions Technical Campus
Team member details
Team Name Artificial Intelligence Based Two Way
Communication System Using Hand
Gestures For Deaf and Mute People
Institute name Guru Nanak Institutions Of Technical Campus
Team members >
1 (Leader) 2
Name P. Harika Sk. Nahaz
Batch 2021-2025 2021-2025
Functionalities of product
• Real-Time Bidirectional Communication: Combines an AI-powered wearable glove with a mobile app for
seamless conversion of gestures to speech/text and vice versa.
• Localized and Inclusive: Supports Indian Sign Language (ISL), local languages, and eliminates the need for
interpreters.
• Affordable and Scalable: Designed to be portable, cost-effective, and adaptable across industries and
environments.
• Hardware: Uses flex, pressure, and IMU sensors, microcontrollers (Arduino/ESP32), and Bluetooth/Wi-Fi for
gesture detection and transmission
• Software: Employs AI frameworks (TensorFlow/PyTorch), speech APIs (Google, Amazon Polly), and
Flutter/React Native for app development
• Licensed Tools: Integrates Google APIs and Amazon Polly with open-source frameworks like TensorFlow and
Flutter.
• The product is highly scalable due to its modular design, allowing easy integration of new templates,
languages, or gestures through software updates and AI model retraining.
Product Specifications
• Hardware: The glove-based system features flex sensors for finger bending detection, pressure sensors to capture
pressure variations, and IMU sensors (accelerometer and gyroscope) for hand orientation and motion tracking. A
microcontroller processes sensor data and communicates via Bluetooth/Wi-Fi modules for seamless connectivity.
• Software: An AI-powered mobile app employs gesture recognition algorithms to interpret sensor data, supports
text-to-speech and speech-to-gesture conversions, and provides a visual display module for text and animated
gestures. It also allows for customizable language integration to suit diverse users.
• Technical and Physical Specifications:
• Glove material: Lightweight, durable fabric with embedded sensors.
• Connectivity: Bluetooth/Wi-Fi, range up to 10 meters.
• Power: Rechargeable battery with 6-8 hours runtime.
• Mobile App: Compatible with Android/iOS, supports multiple languages.
• Processing: AI-based gesture recognition with high accuracy and real-time output.
Product limitations *Optional
• Limited Gesture Recognition Range: The accuracy of gesture recognition may reduce if
gestures are overly complex or not within the designed scope of the system.
• Dependency on Mobile App: Requires a compatible smartphone and the app for full
functionality, which may not be accessible to all users.
• Battery Life: The glove's battery lasts 6-8 hours, which may need frequent recharging
for extended use.
• Connectivity Constraints: Bluetooth/Wi-Fi transmission is limited to a range of 10
meters, which may hinder communication over longer distances.
• Customization Challenges: Adding support for new languages, gestures, or unique
user requirements may require additional development time and resources.
• Environmental Sensitivity: Sensors may be affected by external factors like extreme
temperatures, moisture, or dirt, impacting accuracy and durability.
Architecture *Optional
Glove-Based System:
• Sensors: Flex, pressure, and IMU sensors for gesture detection.
• Microcontroller: Processes sensor data and communicates with the mobile app via Bluetooth/Wi-Fi.
• Communication Module: Bluetooth/Wi-Fi module for data transmission to the mobile app.
Mobile App:
• Data Reception: Receives gesture/speech data through Bluetooth/Wi-Fi.
• Data Processing: AI algorithms process and interpret gestures/speech.
• Output Interface: Displays text or animated gestures; provides speech output.
Cloud Integration:
• Data Storage: Cloud solutions (AWS/Google Cloud) for storing user data, models, and updates.
• API Integration: Real-time interaction with cloud-based speech-to-text and text-to-speech services.
Brief on Programming Module
Programming Languages:
• C/C++ for sensor data processing and microcontroller programming.
• Python (TensorFlow/Keras) for AI-based gesture recognition and machine learning algorithms.
• JavaScript (React Native) or Dart (Flutter) for mobile app development.
• C# for implementing gesture animations with Unity.
Software Modules:
• Sensor Data Processing: Filters and processes sensor inputs for gesture recognition.
• AI Algorithms: Gesture recognition and NLP for speech-to-text conversion.
• Mobile Application: Cross-platform app for communication between the glove and user.
• Text-to-Speech: Uses Google TTS or Amazon Polly for voice output.
• Gesture Animation: Visual representation of gestures using Unity/3D modeling libraries.
Cloud Solutions:
• Integration with Cloud Storage/Services (e.g., AWS, Google Cloud) for storing user data, updates,
and training machine learning models.
Execution plan
Steps from Drawing Board to Prototype:
• Conceptualization: Define project goals, target audience, and features of the communication system.
• Feasibility Analysis: Conduct technical, financial, and market feasibility studies.
• Design Hardware: Develop detailed designs for glove sensors, microcontroller, and communication module.
• Software Architecture: Outline app structure, AI algorithms, and cloud integration for gesture recognition.
• Prototype Development: Create the first functional prototype combining hardware (glove) and software
(mobile app).
• Initial Testing: Validate basic functionalities such as gesture detection, mobile communication, and data
processing.
• Refinement: Optimize the hardware and software for accuracy, reliability, and user-friendliness.
• Prototype Validation: Test the working prototype with a small user group to gather feedback and identify
improvements.