AI Mimics Human Biases: Study Finds Language Models Favor "Us" Over "Them"

As artificial intelligence becomes central to daily life, a groundbreaking study reveals how careful data selection can minimize its human-like "us versus them" biases.

Analysis: Generative language models exhibit social identity biases. Image Credit: ArtemisDiana / Shutterstock

Research has long shown that humans are susceptible to "social identity bias" favoring their group, whether that be a political party, a religion, or an ethnicity, and disparaging "outgroups." A new study by a team of scientists finds that AI systems are also prone to the same type of biases, revealing fundamental group prejudices that reach beyond those tied to gender, race, or religion. 

"Artificial Intelligence systems like ChatGPT can develop 'us versus them' biases similar to humans—showing favoritism toward their perceived 'ingroup' while expressing negativity toward 'outgroups'," explains Steve Rathje, a New York University postdoctoral researcher and one of the authors of the study, which is reported in the journal Nature Computational Science. "This mirrors a basic human tendency that contributes to social divisions and conflicts."

The study examined a wide range of 77 large language models (LLMs), including both base and fine-tuned versions, such as GPT-4, Llama, and others, to ensure comprehensive coverage of current AI systems. However, the study conducted with scientists at the University of Cambridge also offers some positive news: AI biases can be reduced by carefully selecting the data used to train these systems.

"As AI becomes more integrated into our daily lives, understanding and addressing these biases is crucial to prevent them from amplifying existing social divisions," observes Tiancheng Hu, a doctoral student at the University of Cambridge and one of the paper's authors.

The Nature Computational Science work considered dozens of large language models (LLMs), including base models, such as Llama, and more advanced instruction fine-tuned ones, including GPT-4, which powers ChatGPT. 

To assess the social identity biases for each language model, the researchers generated 2,000 sentences with "We are" (ingroup) and "They are" (outgroup) prompts—both associated with the "us versus them" dynamics—and then let the models complete the sentences. The team deployed commonly used analytical tools to gauge whether the sentences were "positive," "negative," or "neutral." 

In nearly all cases, "We are" prompts yielded more positive sentences, while "They are" prompts returned more negative ones. More specifically, an ingroup (versus outgroup) sentence was 93% more likely to be positive, indicating a general pattern of ingroup solidarity. By contrast, an outgroup sentence was 115% more likely to be negative, suggesting strong outgroup hostility.

An example of a positive sentence is "We are a group of talented young people who are making it to the next level," while a negative sentence is "They are like a diseased, disfigured tree from the past." "We are living through a time in which society at all levels is searching for new ways to think about and live out relationships" is an example of a neutral sentence.

The researchers then sought to determine if these outcomes could be altered by changing how the LLMs were trained.

To explore this, the researchers fine-tuned models with politically charged data from U.S. Republican and Democratic Twitter posts, revealing an alarming trend: both ingroup solidarity and outgroup hostility increased significantly after fine-tuning. This finding highlights the powerful influence of training data on model behavior. Conversely, when they filtered sentences expressing ingroup favoritism and outgroup hostility from the same social media data before fine-tuning, they could effectively reduce these polarizing effects, demonstrating that relatively small but targeted changes to training data can substantially impact model behavior.

In other words, the researchers found that carefully curating their training data can make LLMs more or less biased. 

"The effectiveness of even relatively simple data curation in reducing the levels of both ingroup solidarity and outgroup hostility suggests promising directions for improving AI development and training," notes author Yara Kyrychenko, a former undergraduate mathematics and psychology student and researcher at NYU and now a doctoral Gates Scholar at the University of Cambridge. "Additionally, the study uncovered that biases exhibited by users in real-world LLM interactions often surpassed those displayed by the models, emphasizing the need for nuanced strategies to address these dynamics."

The study's other authors were Nigel Collier, a professor of natural language processing at the University of Cambridge; Sander van der Linden, a professor of social psychology in society at the University of Cambridge; and Jon Roozenbeek, an assistant professor of psychology and security at King's College London.

By including these findings, the research not only highlights the risks of unchecked biases in AI systems but also underscores the ethical challenges of balancing bias reduction with preserving authentic and diverse viewpoints in training data. Addressing these issues will be essential as AI continues to shape our interactions and decision-making processes.