Query language pushes LLM survey answers away from a country's baseline

A study by Siddhesh Pawar and colleagues on Llama, Mistral and Gemma found that switching query language moves answers away from a country's baseline, while an explicit country cue closes 30–44% of the gap across 1.6 million open-ended responses. Language cues and explicit country cues are not interchangeable.

• Language cues moved answers away from the country baseline across all tested models and countries
• Explicit country cues closed 30–44% of the gap across 1.6 million open-ended responses
• Gemma moved toward the country baseline; Llama and Mistral moved away for nearly all countries
• The baseline was the model's own responses, not human survey answers from that country