Transparent Vision Algorithms against Stereotypes, with a TEDx Talk!

vision algorithms

I am a researcher in the Social Dynamics Team at Bell Labs Cambridge. I work with a number of amazing creative scientists from different disciplines, like Daniele and Luca. We all have a common goal: understanding human and social behaviour on a large scale, using interpretable computational methods. And I look at how automated image analysis can help with this, by turning images into stories.

For example, computer vision algorithms are powerful tools to read the stories behind people’s faces. The impressive accuracy of face detection and analysis systems allows machines to understand many dimensions of people’s lives by just processing their face pictures: personality, political lean, demographics, emotions and so on.

But in our work, we went a bit further! Have you ever tried to determine a person lifestyle by just looking at his or her pictures? Have you ever looked at a group of people and guessed the kind of places where they like to hang out? Hipster places, creative places, posh places… We designed an algorithm that, by just looking at 25 Foursquare profile pictures of a cafe’s customers, would automatically predict the cafe’s ambiance with an impressive accuracy.

And now the part that I like the most. We also asked a group of students to perform the same task. They looked at the same profile pictures and tried to guess the ambiance of the place where those people like to hang out.

We found that, unlike human raters, our algorithm was not influenced by cultural stereotypes when judging people’s faces! For example, while people associated romantic places with the presence of women, the algorithm did not find any statistical evidence for that, and associated the presence of warm colours in people’s picture with the romantic ambiance.

I want to believe that data can help us discover and, hopefully, counter human stereotypes!

More details in this scientific paper. Oh and we delivered a TEDx talk in Verona earlier this year about all this!

Published in

Leave a reply

Thank you! Your subscription has been confirmed. You'll hear from us soon.

Log in with your credentials


Forgot your details?

Create Account