Computer Vision Invades Public Cameras to Track the People

[Matt]

Jason Sosa was walking down a Shanghai sidewalk when he stumbled onto a digital image. It was projected onto the floor just inside a burger joint — yes, a burger joint — just off the sidewalk, and if you got close, the image changed. It would show you, say, a pile of leaves, and if you walked through it, the leaves would fly away.

This was 2007, and Sosa had never seen anything quite like it. “I was compelled to, well, play with it,” Sosa remembers. “I was engulfed in the technology.”

It was a moment that stayed with him. Over the next few years, he was inspired to build a system that could respond not just to movement, but to the way people looked — their clothes and even their faces. If you spotted a young woman, for instance, you could show her a digital advertisement you wouldn’t show to an older man.

The result is Cara, a system that’s already under test in various retail stores and malls. Fashioned by a team of eight engineers at Sosa’s New York-based startup, IMRSV, this new-age creation uses simple cameras, including web cams, to detect faces and other discerning personal features, and the hope is that businesses can use this information to hone their operations, including, perhaps, tailoring digital advertisements to suit particular types of people.

The tool is just one of several new contraptions that tap cameras in public spaces, using “computer vision” algorithms to track what people are up to — and respond accordingly. Another New York-based startup, Placemeter, is using urban cameras — including street traffic cams — to give the average consumer a way of determining how crowded it might be when they turn up at a local store, restaurant, or some other public place. And though Google has banned the practice, others are looking to build face-recognition applications into Google Glass — those computer-powered specs championed by company founder Sergey Brin.

Naturally, such systems are tagged with that catch-all moniker, The Internet of Things. But Placemeter co-founder and chief operating officer Florent Peyre prefers to think of this phenomenon something closer “The Internet of Places.” As with many so-called Internet of Things creations, the aim is to harness at least of portion of our everyday world using computer networks and other tech. In tapping public cameras, Placemeter hopes to create a “platform of data” that can be used by all sorts of applications, some for consumers and some for businesses.

“We take all these video streams and make them smart, basically — turn them into data,” says Placemeter’s other co-founder, Alex Winter.

The ideas behind these systems are nothing less than intriguing. In touting Cara, the pundits can’t help compare the thing to the ridiculously accurate custom ad system that appears in Steven Spielberg’s science fiction film Minority Report. And when the discussion turns to face recognition and Google Glass, we’re inevitably hit some sort of Terminator analogy. But the truth is that we’re only beginning to approach a world where such science fiction has become reality. Systems like Cara are much simpler — and, due to private concerns, they almost have to be.

Monster Media — a company that builds interactive screens for stores, malls, airports, and other settings — recently tested Cara, and though company CEO Chris Beauchamp says the system outperformed competing contraptions from the likes of Intel, he makes it quite clear that it’s a long way from turning in-store advertising on its head.

“There are a ton of guys running around saying the same thing, which is: ‘Hey, we can change out your content depending on who’s in front of it, put them in gender categories and age categories,’” he says.

“That sounds great, and it works in a room about the size of my conference room, [but] I would never rely on any one software to change the message according to who’s standing in front of a screen. It only takes being wrong one time. If you call a guy a girl or vice versa, you end up on a social network for all the wrong reasons.”

In other words, science fiction is still science fiction. “A lot of people get caught up in Minority Report,” Beauchamp says. “There is never going to be a time when a guy is walking through an airport and signs start talking to him. It’s fucking chaos. People are everywhere. They’re wearing hats, scarves, different facial hair. All that plays into this kind of technology.”

With these public systems, privacy is also a concern, and that’s why outfits like IMRSV and Placemeter stop well short of true face recognition. Cara does what Sosa calls face detection, meaning that it determines what type of face you have, without trying to identify who you are, and he says that no images are stored on the company’s servers.

Meanwhile, Placemeter doesn’t even detect faces. It merely identifies numbers of people. “By nature, we don’t even see faces,” says Alex Winter. “Our images are taken at a distance.”

Jennifer Lynch, a staff attorney with the consumer watchdog the Electronic Frontier Foundation, says that any privacy concerns must be examined on case-by-case basis. “You have to look at each system independently,” she says.

Judging from what they say about their respective systems, IMRSV and Placemeter seem to sidestep any major privacy concerns. “The key for Cara is that they’re doing face detection, not recognition,” says Natalie Fronseca, co-founder and executive producer for the Privacy Identity Innovation tech conference, who is very familiar with Cara. “Jason does privacy by design, and that will help him avoid the adverse consequences that often come with data collection.”

But Lynch says that such a system might still violate your privacy in ways you don’t expect. “It depends on what information they’re collecting,” she explains. “They might just collect gender, but what if they can distinguish race and age? When you put those categories into a system — and you’re using multiple cameras in multiple places — you could get to a point where you could reasonably identify somebody.”

Face recognition via Google Glass is another matter entirely — and that’s why Google has banned the practice. Nonetheless, developers, including an outfit called Lambda Labs, are still looking to create face recognition systems for these compu-specs and perhaps similar hardware.

Inevitably, we will move towards a world where public cameras not only watch us, but software systems actively analyze what we’re doing and what we look like — and actively share this information with businesses and other citizens. The question is how detailed this information will be — and how soon.

Sursa Wired.com