Sixth Sense Technology

[Link], Ashish Bhatiya, Dhaval Thakkar, Ashish Bagade.

[Link], V.J.T.I,
Matunga, Mumbai-400019, India.
cvsashank@[Link]
dkt221190@[Link]
ashish2725@[Link]

I. OVERVIEW Sixth sense has evolved as a technology in the recent past

that has been implemented over multiple applications which I
This paper aims at using ‘Sixth Sense Technology’ system am sure most of us are aware of. For those who are not, here is
to merge the physical world with the digital world. a brief introduction. We have got sixth sense theatres that
make us feel what you see. With respect to the mood of the
Fig. 1. scene the theatre is perfumed; for example let’s say the scene
depicts a rainy season that brings in the smell of the sand into
the theatre to make us feel what we see. Even the seats are
built to move over for more thrill and action.

But arguably the most useful information that can help us

make the right decision is not naturally perceivable with our
five senses, namely the data, information and knowledge that
mankind has accumulated about everything and which is
increasingly all available online.

Although the miniaturization of computing devices allows

us to carry computers in our pockets, keeping us continually
connected to the digital world, there is no link between our
digital devices and our interactions with the physical world.
Information is confined traditionally on paper or digitally on a
screen. ‘SixthSense’ bridges this gap, bringing intangible,
digital information out into the tangible world, and allowing
us to interact with this information via natural hand gestures.
‘SixthSense’ frees information from its confines by
seamlessly integrating it with reality, and thus making the
entire world your computer.

Today we are just machines dependant on other machines

for our day to day work. ‘SixthSense’ allows us to paint the
digital world into our real world. It adds that human touch to
our interactions with the digital world. What if we wish to
change the values of a report or check by how much time the
flights are delayed? We cannot find a computer around us all
Fig. 1 The Six senses
the time. Sixth sense allows us to pull these objects from the
physical world and modify them without the use of a
computer.
We've evolved over millions of years to sense the world
around us. The five human senses include the eye, ear, nose,
tongue and body and we humans are totally dependant on it.
When we encounter something, someone or some place, we
use our five natural senses to perceive information about it;
that information helps us make decisions and chose the right
actions to take.
 Fig. 2 The Camera
II. EVOLUTION OF SIXTH SENSE
B. The Projector :
The development of ‘SixthSense’ was from the rollers of
a simple mouse. Pranav Mistry, MIT Media Lab, 1. The projector projects visual information enabling surfaces
connected the rollers of a simple mouse to his fingers and and physical objects to be used as interfaces
traced the motion of his fingers on the computer screen.
He realised that by adding a camera and a projector to the 2. The projector contains a battery inside with a battery life of
above setup, he could use his hand gestures to do all the three hours.
tasks that one does using a computer.
3. A tiny LED displays the data sent from any smart phone on
any surfaces in view object, wall or person.
III. THE ‘SIXTHSENSE’ DEVICE
(1) We can't really describe SixthSense in a line. Sure, it is a Fig. 3.
system that projects information about what surrounds you
over objects' surfaces, but it's also much more.

(2) Basically, SixthSense is a mini-projector coupled with a

camera and a cell phone which acts as the computer and your
connection to the Cloud; all the information stored on the
web.

(3) It is the combined technology of computer along with cell

phone. It works when a person hangs it on his neck and starts Fig. 3 The Projector
projecting through the micro-projector attached to it. Your C. The mirror :
fingers works like the keyboard as well as the mouse The usage of the mirror is significant as the projector
dangles downward in the direction of the neck.
(3) SixthSense can also obey hand gestures, like in the
infamous Minority Report. However, instead of requiring you D. Smart phone :
to be in front of a big screen like Tom Cruise, Sixth Sense can
do its magic—and a lot more—everywhere, even while you 1. A web enabled smart phone in the user’s pocket processes
are jumping hysteric over Oprah's sofa. the video data.

A. The camera : 2. Other software searches the web and interprets the hand
gestures.
1. It recognizes objects around you instantly, with the micro-
projector overlaying the information on any surface, including E. Coloured Markers :
the object itself or your hand
1. It is at the tip of user’s fingers.
2. It tracks the user’s hand gestures.
2. Marking the user’s finger with yellow, red, blue, green
3. It sends the data to the smart phone. tapes helps the webcam recognize gestures.

4. Then, you can access or manipulate the information using 3. The movements and gestures of these markers are
your fingers. That means, it acts as a digital eye connecting interpreted into gestures that act as an interaction instructions
you to the digital world. for the projected application interfaces.

Fig. 3.
Fig. 2.
 Fig. 3 Color Markers
V. USES AND APPLICATION
IV. HOW IT WORKS

(1) The hardware that makes SixthSense work is a

pendant like mobile wearable interface.

(2) It has a camera, a mirror and a projector and is connected

wirelessly to a bluetooth smart phone that can slip
comfortably into one’s pocket.

(3) The camera recognizes individuals, images, pictures,

gestures one makes with their hands

(4) Information is sent to the Smartphone for processing.

(5) The downward-facing projector projects the output image

on to the mirror.

(6) Mirror reflects image on to the desired surface.

(7) Thus, digital information is freed from its confines and

placed in the physical world.
A. Easy to Draw:
A person can draw anything on paint with the help of his
fingers. So, why to use mouse for painting?

B. Capture photos with fingers:

Why to take camera on your holiday as this Sixth Sense
computer will work like your camera. It captures the
photo, when you make a square with your fingers,
highlighting which one you want to frame.

C. Phone Call:
You can call your friend by typing the numbers on your
hand. It displays the keypad of the phone over your palm
and the keys appear on the four fingers. Use your other
hand's finger to press the keys.
D. Check the brand of the Product :
K. Zooming Feature :
It helps you to choose the best brand product from the
super market. The user can zoom in or zoom out using intuitive hand
movements.
E. Read Books easily:
Check out the ratings of the Book you are going to buy, it
checks the ratings from the internet. And another amazing
thing is that it reads the book for you.

F. Newspapers:
Did you see the moving pictures of the Newspaper in the
movie Harry Potter, this is quite similar to it. It Searches
the most appropriate video from the web by seeing the
headlines or the caption of the News report.

L. Feed information on people :

G. Check your Flight Status:
The system will project relevant information about a
You can check the status of the flight while you are in the person such as what they do, where they work, and so on.
Taxi. Just place the ticket in front of the projector and it
checks its status from the internet.

H. Check time :
Draw a circle on your wrist to get a virtual watch that
gives you the correct time.

I. Call up a map :
With the map application we can call up the map of our
choice and then use thumbs and index fingers to navigate
the map.

J. Create multimedia reading experiences:

Sixth Sense can be programmed to project related videos
onto newspaper articles you are reading.
 VI. ADVANTAGES VIII. FUTURE

(1) Clearly, this has the potential of becoming the ultimate

(1) Portable "transparent" user interface for accessing information about
everything around us. If they can get rid of the colored finger
(2) Supports multi touch and multi user interaction. caps and it ever goes beyond the initial development phase,
that is. But as it is now, it may change the way we interact
with the real world and truly give everyone complete
(3) Connectedness between word and information. awareness of the environment around us.

(4) Cost effective. (2) In addition to commercial use it has social uses also. The
SixthSense is being developed as a tool for helping blind and
(5) Data access directly from machine in real time. deaf people realize the world around them by their gestures.
As they have lost one of their senses it will be a great tool for
(6) Mind map the idea anywhere. helping them perceive this world like everyone else.

(7) It is an open source.

REFERENCE
VII. CONCLUSION
[1] [Link] [Link]/projects/sixthsense
(1) Sixth Sense recognizes the objects around us, displaying [2] [Link]/wiki/sixthsense
information automatically and letting us access it in any
way we need.
[3] [Link]/pranavmistry

(2) The Sixth Sense prototype implements several

[4] [Link]/sixth-sense-technology
applications that demonstrate the usefulness, viability
and flexibility of the system.

(3) Allowing us to interact with this information via natural

hand gestures.

(4) The potential of becoming the ultimate "transparent"

user interface for accessing information about everything
around us.