SIXTH SENSE TECHNOLOGY

BY MANMOHAN MEHTAAASHISH RATHORE

GAURAV GUPTAPANKAJ KUMAR

INTRODUCTION

“Sixth Sense is a wearable gestural interface device that

augments the physical world with digital information and

lets people use natural hand gestures to interact with that

information. ”

It was developed by PRANAV MISTRY,

COMPONENTS

• A pocket projector,• A mirror,• A camera.

• The projector: projects visual information, enabling surfaces, walls and physical objects around the wearer to be used as interfaces;

• The camera and hands: recognizes and tracks the user's hand gestures and physical objects using computer-vision based techniques.

• The software program: processes the video stream data took by the camera and tracks the locations of the colored markers (visual tracking of colored fingers) at the tip of the user’s fingers using simple computer-vision techniques.

• The maximum number of tracked fingers is only constrained by the number of unique fiducials, thus Sixth Sense also supports multi-touch and multi-user interaction.

OVERVIEW

Applications

The Sixth Sense prototype implements several applications that demonstrate the usefulness, viability and flexibility of the system.

• The map application

• The drawing application

• freehand gestures (postures)

• Sixth Sense also lets the user draw icons or symbols in the air using the movement of the index finger and recognizes those symbols as interaction instructions.

For example, drawing a magnifying glass symbol takes the user to the map application or drawing an ‘@’ symbol lets the user check his mail.

The gesture of drawing a circle ‘o’ on the user’s wrist projects an analog watch to effectively tell time & the gesture of minus sign ‘-’ removes the clock.

• The Sixth Sense technology also augments physical objects the user is interacting with by projecting more information about these objects onto them.

• Eg., a newspaper can show live video news.

USES

PICTURES

PICTURES

PICTURES

PICTURES

PICTURES

HOW IT WORKS???

The software utilizes special made 2d prints in order to track where the virtualization begin .

2d prints are the optimal choice due to their generous error correcting built into the images, Since the software has the image preloaded in, once it is found in the image provided by the camera it can extrapolate in real time the size, angle, and location the virtualization should occur in order to look natural in the photo.

TECNIQUE BEHIND

HOW SOFTWARE WORKSThe software works on the basis of computer vision. A small camera acting an eye, connecting us to the world of digital information. Processing is happening in the mobile phone, and basically works on computer vision algorithms. Approx 50,000 lines of code are used. The software recognizes 3 kinds of gestures:• multitouch gestures, like the ones we see in the iPhone – where we touch the screen and make the map move by pinching and dragging.•freehand gestures, like when you take a picture or a namaste gesture to start the projection on the wall. • iconic gestures, drawing an icon in the air. Like, Whenever we draw a star, show us the weather. When I draw a magnifying glass, show me the map.

This system is very customizable. We can make our own gesture which our sixth sense device can understand. We can change the Sixth Sense to our need.

SIXTH SENSE IN GAMING

We can do all the kinds of gaming that exists now, but not only that, we can use the physical world inside the game. You can play with physical stuff, invent some new games. Maybe you can hide something in the physical world -- open a book and hide something in the pages.

COST

The current prototype system costs approximately $350 to build, mainly due to the micro-projector. The software may be available for free on the model of open and editable freeware

LIMITATIONS

• Software does support the ability to use real time video streams in order to produce augmented reality.

• Hardware limitations of the devices, that we currently carry around with us.For example many phones will not allow the external camera feed to be manipulated in real time.

• Post processing can occur however.

FUTUR E 0F SIXTH SENSE

Interactive Advertisements. True 3d print media. 3d visualizations. Solar batteries via small solar panel. Camera can act as a third eye for the

blind person

“According to researchers, after 10 years we will be here with the ultimate sixth-sense brain implant.”

CONCLUSION

THANKS

F.A.Q