Retrieving information from the Web when you're on the go can be a challenge. To make it easier, graduate student Pranav Mistry has developed SixthSense, a device that is worn like a pendant and superimposes digital information on the physical world. Unlike previous "augmented reality" systems, Mistry's consists of inexpensive, off-the-shelf hardware. Two cables connect an LED projector and webcam to a Web-enabled mobile phone, but the system can easily be made wireless, says Mistry.
Users control SixthSense with simple hand gestures; putting your fingers and thumbs together to create a picture frame tells the camera to snap a photo, while drawing an @ symbol in the air allows you to check your e-mail. It is also designed to automatically recognize objects and retrieve relevant information: hold up a book, for instance, and the device projects reader ratings from sites like Amazon.com onto its cover. With text-to-speech software and a Bluetooth headset, it can "whisper" the information to you instead.
Remarkably, Mistry developed SixthSense in less than five months, and it costs under $350 to build (not including the phone). Users must currently wear colored "markers" on their fingers so that the system can track their hand gestures, but he is designing algorithms that will enable the phone to recognize them directly. --Brittany Sauser
1. Camera: A webcam captures an object in view and tracks the user's hand gestures. It sends the data to the smart phone.
2. Colored Markers: Marking the user's fingers with red, yellow, green, and blue tape helps the webcam recognize gestures. Mistry is working on gesture-recognition algorithms that could eliminate the need for the markers.
3. Projector: A tiny LED projector displays data sent from the smart phone on any surface in view--object, wall, or person. Mistry hopes to start using laser projectors to increase the brightness.
4. Smart Phone: A Web-enabled smart phone in the user's pocket processes the video data, using vision algorithms to identify the object. Other software searches the Web and interprets the hand gestures.
Credit: Sam Ogden
Massachusetts Institute of Technology
Comments
Netizen
08/20/2009
Posts:14
srsubaak
08/27/2009
Posts:1
Why not add all of these up one day?
- Augmented Reality & AI
- Project Natal Reality Recognition
- Ubiquitous Wireless Fast Internet Access
- GPS technology
- Glasses or Contact Lenses with all of the above
You'd be standing in a corner, just by looking at a street sign you could have a hovering handle next to it, with your fingers you could touch it or pull it, and have a few options, like:
- Showing a map
- Points of interests in the area
- Historic Facts
- Bus Schedules
The same would happen with everything that the Augmented Reality & AI could recognize. If you could tell it to keep track of things like your keys, your wallet, your tv remote controls, next time you don't know where they are you could ask your glasses to show you the last known moment where you saw them.
The integration with the internet would be amazing, you could take pictures with hand gestures or voice commands, have them uploaded.
The next iteration of the information society will be amazing, let's hope there are dreamers with more technical and financial capabilities than mine to make these dreams come true.
Angel Leon
gubatron
08/27/2009
Posts:2
gulamriyaz
08/28/2009
Posts:2
gulamriyaz
08/28/2009
Posts:2