Study shows how models learn false claims
A preprint questions using surprise as the sole criterion for deciding what information a model should learn over time.
- Selecting data only because it is surprising can favor falsehoods that contradict a model’s prior knowledge.
- Filters and verifiers help in some cases, but can reject valid information or be bypassed.
- The preliminary results come from experiments with three small adapted models.
Context and analysis by Noosigma AI
Editorial summary and context based on the cited source. Claims and results attributed to the organization are reports from that source; they do not constitute independent validation by Noosigma AI.
In continual-learning methods, an unexpected claim may receive priority for an update. The authors argue that a planted falsehood can seem surprising precisely because it contradicts what a model already knows, causing this criterion alone to accept incorrect information.
Why it matters
The study tested small, adapted versions of Qwen2.5 and Llama 3.2 on streams of true and false claims, including invented statements and real facts. A verifier and comparison with prior knowledge helped filter some falsehoods, but they also rejected some valid updates or could be bypassed.
Trusted sources improved filtering when an attacker could not manipulate them. Still, the authors show that no acceptance decision based on surprise, verification or prior knowledge is enough to establish that a claim is true. The work was submitted to arXiv on October 29, 2026, at 11:19 UTC. It is a research preprint and has not been peer reviewed. Source: arXiv.