This Webpage Lets You Use Jedi Mind Tricks to Scroll Hands-Free
Daniel Rapp has developed a way to use your computer’s mike and speakers to measure the Doppler effect of you waving your arm around.
The Doppler effect is a scientific term for the change in frequency of a wave as the source moves closer to the observer.
That is an esoteric definition (which doesn’t actually tell you anything), so let me explain it a different way: Did you ever listen to the song of an approaching ice cream van and notice that the pitch increased as the vehicle got closer, and decreased as the vehicle departed?
That is an example of the Doppler effect in action.
Rapp has found a way to use the speakers and microphone on your computer to measure the Doppler effect of you waving your arms around. Essentially, he has turned your computer into a low-cost sonar rig; the microphone plays a high pitched sound, and the speakers pick it up. The web page then tracks the changes in the sound (the Doppler effect), and uses that to figure out what you’re doing with your arms.
He’s built a webpage to show that this idea works, and can be used for scrolling and other gestures. The page doesn’t work for me, but I’ve heard from other people that it worked for them (thanks, Sue!).
If it doesn’t work for you, here’s a demo video from Microsoft Research. They first identified this trick back in 2012:
A team of researchers at Microsoft were the first to figure out how to use a computer’s speakers and microphones to measure the Doppler effect and let users interact with their computers simply by waving their hands around. They published a paper back in 2012, and they posted the above video.
The researchers successfully identified a wide range of gestures, including scrolling, taps, pinches, and rotation, but Gizmodo says that the project was never really seen as anything more than a "supporting act" for Microsoft’s Kinect sensor.
I would bet they’re wrong on that. This trick may have arrived to late to be used in Kinect, but I would bet that the idea will be used in the next generation Kinect – or one of its competitors.
I don’t think this trick is good enough at tracking motion on its own, but it’s still worth adding to a motion tracker. If that device already has a mike and speakers, I think it would be a good idea to use them for one additional purpose.