Scientists reconstruct speech by watching a bag of potato chips

[nedo]

Scientists reconstruct speech through soundproof glass by watching a bag of potato chips.

Subtle vibrations can be translated back into audio

Your bag of potato chips can hear what you're saying. Now, researchers from MIT are trying to figure out a way to make that bag of chips tell them everything that you said — and apparently they have a method that works. By pointing a video camera at the bag while audio is playing or someone is speaking, researchers can detect tiny vibrations in it that are caused by the sound. When later playing back that recording, MIT says that it has figured out a way to read those vibrations and translate them back into music, speech, or seemingly any other sound.

While a bag of chips is one example of where this method can be put to work, MIT has found success with it elsewhere, including when watching plant leaves and the surface of a glass of water. While the vibrations that the camera is picking up aren't observable to the human eye, seemingly anything observable to a camera can work here. For the most part the researchers used a high-speed camera to pick up the vibrations, even using it to detect them on a potato chip bag filmed 15-feet away and through a pane of soundproof glass. Even without a high-speed camera though, researchers were able to use a common digital camera to pick up basic audio information.

"We’re scientists, and sometimes we watch these movies, like James Bond, and we think, ‘This is Hollywood theatrics. It’s not possible to do that. This is ridiculous.’ And suddenly, there you have it," Alexei Efros, a University of California at Berkeley researcher, says in a statement. "This is totally out of some Hollywood thriller. You know that the killer has admitted his guilt because there’s surveillance footage of his potato chip bag vibrating." The research is being described in a paper that will be published at the computer graphics conference Siggraph.

Sursa: aici

[Nytro]

[h=1]Focus Your Ears with The Visual Microphone[/h]

August 6, 2014 By Adam Fabio Leave a Comment

A Group of MIT, Microsoft, and Adobe researchers have managed to reproduce sound using video alone. The sounds we make bounce off every object in the room, causing microscopic vibrations. The Visual Microphone utilizes a high-speed video camera and some clever signal processing to extract an audio signal from these vibrations. Using video of everyday objects such as snack bags, plants, Styrofoam cups, and water, the team was able to reproduce tones, music and speech. Capturing audio from light isn’t exactly new. Laser microphones have been around for years. The difference here is the fact that the visual microphone is a completely passive device. No laser or special illumination is required.

The secret is in the signal processing, which the team explains in their SIGGRAPH paper (pdf link). They used a complex steerable pyramid along with wavelet filters to obtain local pixel motion values. These local values are averaged into a global motion value. From this global motion value the team is able to measure movement down to 1/1000 of a pixel. Plenty of resolution to decode audio data.

Most of the research is performed with high-speed video cameras, which are well outside the budget of the average hacker. Don’t despair though, the team did prove out that the same magic can be performed with consumer cameras, albeit with lower quality results. The team took advantage of the rolling shutter found in most of today’s CMOS imager based consumer cameras. Rolling shutter CMOS sensors capture images one row at a time. Each row can be processed in a similar fashion to the frames of the high-speed camera. There are some inter-frame gaps when the camera isn’t recording anything though. Even with the reduced resolution, it’s easy to pick out “Mary had a little lamb” in the video below.

We’re blown away by this research, and we’re sure certain organizations will be looking into it for their own use. Don’t pull out your

yet though. Foil containers proved to be one of the best sound reflectors.

Thanks [Zach]!

Sursa: Focus Your Ears with The Visual Microphone