Modeling human behavior with Airbnb

With just a few clicks on TripAdvisor or Airbnb, you can book a romantic apartment for a weekend away with your partner, or a stylish restaurant for a business lunch. The rapid decisions involved, based mainly on images, are far from trivial given their commercial importance and the economic revolution represented by the advent of on-demand economy websites like Airbnb. But what is it about an image that prompts us to describe an interior as “trendy”, “colorful” or “practical”? To answer that question, researchers at Idiap Research Institute and at EPFL have been working with psychologists from the University of Lausanne. They want to gain a better understanding of social media users’ perceptions and behavior and then use this knowledge to program computers capable of making decisions in a more human way. “In the era of big data, machines are increasingly behind a large number of decisions,” explains Daniel Gatica-Perez, adjunct professor at EPFL School of Engineering and Digital Humanities Institute. “Our aim is to make them as similar as possible to human decisions.”

A collaboration between psychologists and engineers

To understand how a first impression is formed, researchers first carried out interviews with guests and travelers, asking them how they select accommodations. They used 350,000 images of 22,000 properties listed on Airbnb in Switzerland and Mexico, and applied to them an algorithmic analysis to check that they were images of interiors. They then selected 200 properties at random and sent a list of adjectives to online observers. Those observers had to decide how accurately the adjectives described each property, on a scale of 1 to 7. Some adjectives were more factual (such as “clean” and “cluttered”), while others were more subjective (such as “bohemian” and “charming”). That stage, carried out in collaboration between psychologists and engineers, revealed which characteristics all participants agreed on and which ones they disagreed on. For properties described as “colorful” or “dark,” most respondents agreed with those adjectives and the scores were very similar. Scores for other adjectives, such as “relaxed” or “traditional,” varied widely depending on the property.

Analyzing human perception online

The scientists then carried out modeling based on the data obtained. They tried to detect which characteristics of the photos prompted the participants to describe them using a given adjective, in order to program computers to recognize them. Next, they looked at the extent to which the adjectives were interrelated. Will people describing a property as “colorful” also associate the adjective “clean” with that property? What is the connection between “pretentious, “trendy,” “organized” and “large”? How are positive and negative adjectives, and factual and subjective adjectives, interrelated? And why is the adjective “romantic” more closely associated with “sophisticated” than with “trendy”? “We might expect ‘large’ and ‘spacious’ to be very close together in people’s minds, and ‘cluttered’ and ‘empty’ to be very far apart,” says Gatica-Perez. “But the relationships are more complex. Using our system, if we recognize one characteristic, we can also associate other adjectives connected to them in people’s minds.”

Machines helping humans

Finally, the researchers took the property images and applied algorithms used in the field of deep learning, comparing the results with those obtained from humans. Eventually, professionals such as architects or designers could apply the results to photos of interiors. The laboratory is also monitoring the development of image sharing sites which, for a given place, display very different photos – professional and amateur – leading to widely varying perceptions. However, the scientists’ main goal is to understand the characteristics of images and the connections that determine the way we form impressions, so that they can program computers to imitate them. “We often hear that machines perform better than humans,” concludes Gatica-Perez. “Our aim is different: we want to train machines to recognize these subtleties that humans perceive and express in their day-to-day lives, and to use them to support people’s real needs.”

“Check Out This Place: Inferring Ambiance from Airbnb Photos”, Laurent Son Nguyen, Salvador Ruiz Correa, Marianne Schmid Mast and Daniel Gatica-Perez. IEEE Transactions on Multimedia.

This study was carried out in Switzerland and Mexico as part of the EPFL-UNIL Collaborative Research on Science and Society program.