Study Shows AI Models Less Accurate in Predicting Public Opinion in Chile Compared to the U.S.

Original article: Estudio evidenció que modelos de IA tienen menor precisión en predecir la opinión pública en Chile que en EEUU

Can artificial intelligence predict what we think? This question was posed by Andrés Abeliuk and Vanessa Gaete from the Faculty of Physical and Mathematical Sciences at the University of Chile, along with Naim Bro, a professor at the School of Government at the Adolfo Ibáñez University (UAI), in the study Auditing socio-demographic and cross-societal fairness in LLM-simulated public opinion.

The research aimed to determine how well artificial intelligence models can predict public opinion responses based on demographic information.

The findings revealed a significant difference: the models achieve greater accuracy in predicting public opinion responses in the United States than in Chile. «And not because of inherent biases, but because they learn from data that may reflect real-world inequalities,» explains Professor Abeliuk.

According to the researcher, a potential explanation lies in the data used to train these models: much of it originates from the U.S. context, making it harder for the models to accurately represent other cultural and social realities, such as that of Chile.

«Moreover, in the context of Chile, we observe that the models make more errors in representing women, older individuals, religious people, and those with lower education levels,» comments Abeliuk.

This, adds the academic from the University of Chile, could reflect existing inequalities in the data used to train the models, where some groups might be underrepresented or have less available information: «When these factors combine, the differences can become even greater,» points out the specialist.

The results, therefore, raise a pertinent question for the development and use of artificial intelligence: How well can AI represent a society that is not sufficiently present in the training data?

For the research team, moving towards truly useful and fair artificial intelligence systems requires considering this diversity.

«Overall, these results show that for artificial intelligence to be genuinely useful and just in our societies, we need to develop models that are more inclusive and adapted to the cultural and social diversity of countries like ours,» concluded Professor Abeliuk.

We will continue to provide updates.