AI models ‘weak and fallible’ on climate science, UN-backed study reports

Artificial intelligence models can quickly turn into climate crisis deniers at the mildest challenge from users, according to a new UN-backed study.

While AI models can accurately confirm simple facts about climate change, eight times out of 10 when they pivoted to a wrong answer, they did so "at the very first sign that the user mildly disagreed" to confirm incorrect statements such as "CO2 is not the main driver of climate change".

The report uncovered a strong tendency for AI models to be sycophantic, meaning the model is more interested in pleasing the user than standing by facts – an issue identified when people turn to AI chatbots for personal advice.

The report from Unesco and University College London was launched at the UN climate summit in Turkey on Monday, revealing the results of tests on 15 of the world's most popular AI models.

Leona Verdadero, Unesco's lead author of the report, said: "AI is not neutral. When it comes to how it responds to the user, it will be pushed to align with the users' preference instead of sticking up for the facts. The fact they could so easily abandon truth and be more agreeable to a user's [misinformation] shows it is weak and fallible."

When AI was asked how a single climate policy would work, such as a carbon tax, the models were mostly accurate. But when asked how two climate policies could work together, accuracy collapsed to no better than random guessing, the researchers said.

Dr María Pérez-Ortiz, of UCL's department of computer science and a senior author of the report, said: "AI systems are remarkably good at explaining scientific knowledge, but this can give us a false sense of their capabilities. Our findings show why we must be careful when moving from asking AI what we know about climate change to asking what we should do about it."

A white Cop31 sign in a green space at the event area with palm trees in the background against a blue sky.

AI models also had a language problem. Accuracy was scored at 88% for people using English, and comparatively high in Spanish, French and Portuguese.

Responses were less accurate for people speaking Hindi, Bengali and Urdu, and the worst for Swahili speakers, at 78%. This was particularly important, the report authors said, because those languages were spoken in places among the hardest hit by the climate crisis.

The report was published on the same day Turkey, the Cop31 host, launched the AI Antalya pledge to make AI "part of the climate solution" with datacentres that are "sustainably" powered.

Julie Cerqueira, of the Natural Resources Defense Council, said AI's rapid growth was "raising a new set of environmental and existential concerns around the world".

She said the pledge was an important start but "more work is needed, and it needs to move quickly to keep up with the industry's rapid-fire development and goldrush mentality".

Simon Stiell, the UN's chief climate negotiator, said AI that "spreads disinformation, undermines science, divides societies and concentrates wealth" would damage the public trust the industry would rely on to grow.

"Communities are already pushing back against datacentres, concerned about their water, their energy bills and their environment. So public support is already sinking in deeper underwater," he said.

Original source AI models ‘weak and fallible’ on climate science, UN-backed study reports

Back to home