Article: Facial analysis in emerging markets

One of the latest innovations in market research is facial and emotional analysis using artificial intelligence. This technology employs recognition software to detect emotional insights from facial expressions, which can enhance research by delving further into respondents’ reactions to healthcare communications or other stimuli. Complementary tools can be used to elicit additional meaning from tone of voice, through to laughter and anger.

As a company, we have been successfully using this technique in both qualitative and quantitative research settings to test communications such as disease campaigns and e-details. Indeed, our company has won several awards for its use in research conducted with Janssen. As specialists in emerging markets, we wanted to see if the technique has utility in emerging markets.

As a rule of thumb, it is generally accepted that facial expressions are broadly the same across countries and cultures. For example, smiling donates happiness and frowning shows sadness. Research conducted by psychologist Paul Ekman has shown both western cultures and preliterate communities in Africa communicate using the exact same facial expressions for happiness, disgust and contentment.

Facial analysis used in conjunction with traditional methods can provide a powerful tool to understand our patients, consumers and HCPs

There has, however, been debate over the extent to which facial expressions are innate verses cultural. New thinking has challenged the classical universality of human emotions. The theory is that facial expressions and corresponding emotions are cultural, and learnt with respect to an individual’s surrounding environment. Research by scientists from the Institute of Neuroscience and Psychology at the University of Glasgow suggests that although a similar facial expression can indicate an emotion, the interpretation of facial expressions may differ between regions and cultures. Chinese participants for example appear to focus on the eyes to interpret expressions, whereas western Caucasians tend to focus on the mouth. In Brazil, smiling for longer is a sign of expressing happiness, and in Japan a smile is often a way to communicate politeness rather than happiness necessarily.

AI technology for facial analysis uses two techniques to interpret emotions. Firstly, the computer vision precisely captures facial expressions and secondly, the machine-learning algorithm analyses and interprets the emotional content picked up from the computer vision. One implication for any market research technology is that certain cues may get missed if the learnt behaviour from the algorithm that the programme is based on does not include enough faces from diverse regions.

A major criticism of some facial analysis technology is the racial bias due to a limited number of darker skinned faces included in product development. One infamous case is when the software ‘Rekognition’ owned by the tech giant Amazon, wrongly classified African American women as men. Other studies have shown that negative expressions are more likely to be assigned to black men verses white Caucasians. As market researchers, and users of this technology we therefore need to ask ourselves how the algorithms are built, and if there are any racial biases. It is vital that the learned behaviour of the technology is truly diverse and based on people of all skin colours and cultures (including wearers of hijabs or burkas).

Our partner Affectiva has the largest global database of facial expressions – 7.5 million faces from 87 countries, including many from Asia and South America. They tell us “the wealth of data we have in many countries provides so much training material for our deep learning algorithms that they are human level accurate and allow us to handle cultural differences. While all humans tend to emote the same way, when, why and how we emote is strongly linked to the culture we live in. Our software takes account of those cultural differences.”

We believe that, working in partnership with Affectiva, facial analysis used in conjunction with traditional methods can provide a powerful tool to understand our patients, consumers and HCPs in many cultures and regions, so long as it is in markets where there is sufficient data for analysis without bias.