In an article recently published in the journal PLOS, researchers discussed an innovative project that used observation attributes of humans to recognize facial expressions and predict the behavior of a person.
Study: Human vs. Machine Facial Expression Recognition: A Study on Personality and Observational Behavior. Image credit: Andy Shell/Shutterstock
The study examined the recognition of facial expressions by humans using a limited set of facial attributes, which is an approach used in machine learning. The researchers built a Bayesian model that studied how personality traits and observational behaviors influence the identification of facial expressions.
Background
Expressions are classified into six different types used in varied fields of study. These methods help humans evaluate a person’s social cognition and the person’s nature and relations with fellow human beings.
Machine learning techniques for facial expression recognition have flourished, often relying on 2D facial landmarks, but their accuracy varies compared to human recognition, especially for negative expressions. Although machines are as good as humans at recognizing happy and surprised facial expressions, they struggle to identify sad or fearful expressions accurately because they are trained using a limited number of facial images and only use fewer facial landmarks compared to humans. The relationship between facial features and observational behavior has gained attention, with studies using eye tracking showing different gaze patterns for different expressions.
Existing research has shown a strong link between people's personality traits and how they observe things, even when their eye movements are controlled. This study suggests that there are two types of observational behavior: conscious (controlled) and unconscious (uncontrolled). So far, no study has looked at how conscious observational behavior varies when people recognize facial expressions, especially using eye tracker data. In other words, researchers have discovered that personality traits influence how people recognize things in both conscious and unconscious ways. Fortunately, this study does not examine how these factors influence people's ability to recognize facial expressions.
About the Study
"Observing Behavioural Attributes Using Facial Recognition” describes the detailed research involved in the recognition of emotional facial expressions and their relationship with personality traits and observational behavior. This procedure has two main parts and three analyses:
In Part 1, participants watched animated gifs of emotional facial expressions and chose which emotions were displayed. They could access the videos as many times as they wanted and rate their confidence in their choice on a 7-point scale. In Part 2, participants performed the same task but could only see the area around the mouse cursor, limiting their information about the faces.
After both procedures, researchers have insights into different facial features in emotion recognition and how people prioritize information when making emotional judgments. After this process, three types of analyses were performed.
- Analysis A studied the relationship between participants' observational behavior and facial expression recognition.
- Analysis B explored the relationship between participants' personality traits and their choice of facial expressions.
- Analysis C researched the relationship between participants' personality traits and their observational behaviors when judging facial expressions.
Tools used in data analysis were the following:
- Bayesian hierarchical models
- Rstan for parameter estimation.
- Weakly informative priors for fixed and random effects.
- Markov Chain Monte Carlo (MCMC) sampling convergence
The video stimuli used by the researchers comprised 68 landmark points representing an array of facial expressions.
Results/Discussions
The study investigated if humans could correctly recognize facial expressions using a limited number of facial landmarks and examined differences in conscious observational behavior when recognizing various facial expressions.
The study found that personality attributes significantly contributed to inaccuracies in facial expression recognition. Participants with high Agreeableness were more likely to recognize happiness, while those with high Extraversion were likely to correctly identify surprise. Conversely, individuals with high openness tended to confuse anger as sadness and fear as disgust, while those with high conscientiousness are likely to identify surprise as anger incorrectly.
The study also found that restricting participants' observational behaviors had a significant impact on their ability to recognize facial expressions. They were less accurate in recognizing negative expressions, and the tendency to misidentify fear as a surprise was more pronounced. Additionally, people with high Extraversion generally did not select negative expressions.
Overall, the study provides valuable insights into the limitations of human facial expression recognition and the role of personality attributes in this process. The findings also suggest that restricting observational behaviors can have a significant impact on the ability to recognize facial expressions accurately.
Conclusion and Future Scope
To summarize, the study aimed to explore human recognition of facial expressions using limited landmark representations and examine the impact of conscious observational behaviors. Results revealed that humans can accurately recognize positive expressions, and the way people behave when they observe facial expressions is influenced by their personalities. Sometimes our unconscious observations can interfere with making accurate judgments.
The findings agreed with that of previous studies on the use of machine learning for facial expression recognition. Even with minimal information on facial expression, humans were able to recognize positive expressions, but they struggled with identifying negative expressions. However, humans in this study more accurately recognized sadness compared to machine learning systems.
Additionally, examining the practical applications of this research, such as in human-computer interaction or mental health diagnosis, is a promising avenue. Using these observational behavior dynamics can not only help understand the mechanism of facial recognition but also improve the accuracy of human facial recognition applications in machine learning. Interdisciplinary collaboration between psychology, neuroscience, and technology fields will be crucial for a comprehensive understanding and application of facial expression recognition.
Using Machine Learning to Identify Suicide Risks