Nuno Moniz (University of Notre Dame) in collaboration with Kids Konnect Ukraine
Over 400 conversations were conducted by 25 young people (12-16 years old) with ChatGPT, Claude and Qwen, in
English and in Ukrainian. These conversations were about Holodomor, the Euromaidan Revolution, the annexation of
Crimea, and the ongoing war with the Russian Federation. Every answer was audited, leading to this analysis.
This is not a simulation of what children might be shown: It is a record of what they were shown.
Key Findings
Almost all AI safety work on disinformation asks if a system is fetching material from hostile sources. Here, the
answer was mostly no. The problem sits one layer down: we show that contested framings are already built into
these systems from the data they were trained on, and they come out in the model’s own voice, with no source
attached for anyone to check.
- On the war in Ukraine, the pattern is dramatic, and it is not coming from web searches.
Framings that echo Russian talking points, such as NATO enlargement presented as a cause of the war rather than as
Moscow’s stated pretext, appeared roughly seventeen times more often in conversations about the current war than in
conversations about the annexation of Crimea with all three chatbots showing it independently. Meanwhile, the network built specifically to spread propaganda concerning the war rarely appeared. The propaganda is not being fetched. It is already in the models. In one chatbot’s English answers, every single unsourced aside about why the war started elevated NATO expansion. Not one elevated any of the causes historians actually emphasize. Eight out of eight, in the connective sentences the model wrote itself, mostly in conversations where it had searched for nothing. No source to check, no citation to flag, nothing for a filter to catch.
- Presence of hostile sources is rare but still concerning.
One tool handed children working links to RT and Sputnik. Another cited the academic proponent of the “false flag sniper” theory as a bare name, with no title and no warning. A Russian government domain supplied part of one answer on the causes of the famine; an SEO content farm underpinned another answer’s list of key figures. None of this is visible in the text
a child reads. Rare does not mean harmless: it means hard to find.
- What is left out matters as much as what is put in.
One tool, asked in Ukrainian about the Holodomor across nineteen conversations, never once mentioned the continued grain exports or the refused foreign aid: the evidence showing the starvation was chosen, not merely suffered. Harms get described with nobody performing them. Every one of these answers is accurate. Each leaves a child less able to say who did it, and why.
- Hallucinations in an authoritative tone.
Invented books, invented authors, invented quotes attributed to real institutions. Holodomor death tolls ranged from 3.5 to 15 million across the study. Nothing in the tone, formatting or confidence distinguishes the fabricated passages from the sound ones, and errors lean toward what the reader already expects, so the false parts are the least likely to prompt a question.
What Follows for Policy
- Stop treating this as a source problem. Blocking hostile sources addresses the layer these systems already handle well.
The material that reaches children is generated, not fetched, and no source filter can see it. Oversight has to reach training
data and model behaviour, not just the links to sources used to generate answers in conversations. - Report what models generate on contested topics. Unprompted, unsourced framing is not currently measured. It is
measurable (the object of our study) and it varies across topics by a factor of seventeen. - Require tools to show whether they searched. Answers written without any lookup carried most of the invention and most of the tilted framing. Users cannot see the difference, and it is the best single predictor of reliability available today.
- Judge these tools on completeness, not just on falsehood. Current safety screening asks whether a system said
something untrue or cited something hostile. Neither question catches a system that simply leaves out the strongest
evidence of intent, and in our study, several did. - Children should not use these tools as sources for contested history. Reliability in this study came only from reading many conversations, in both languages, and checking every citation. Research tells us that children are prone to believe the
authoritative tone of chatbots. No tool and no language was safe to read alone.