Surprise Alone Learns Lies: Evidence-Gated Continual Learning for LLMs from Streaming Claims

⚡ In one sentence AI-generated · check the original

Key findings

• The original model's own prior separates lies about known facts from true new facts (AUC 0.78-0.83) but not from legitimate updates (AUC 0.53), saving 43% of judge calls.
• A verifier-key judge removes lies only as accurately as its verifier, and real LLM judges reject unsourced truths and source-backed updates.
• Routing contradicted claims to trusted sources yields the lowest poison rate, but if an attacker plants matching sources for 30% of lies, the system is worse than the judge alone.

📄 Abstract

Continual learning methods for large language models increasingly decide what to learn by surprise, keeping the inputs the model finds least predictable. This rule is unsafe when the stream contains lies. A planted falsehood about something the model knows has low probability under the model's belief, so it is highly surprising by construction, and a surprise-driven learner absorbs nearly every such lie, at a rate similar to random selection with the same budget. We study the problem with LoRA-adapted students (Qwen2.5-1.5B, Qwen2.5-3B and Llama-3.2-3B) that learn from a stream of true and false claims about invented people and real CounterFact facts. We report four findings. (1) A verifier key, a judge that must approve each claim, removes lies to the extent that its verifier is accurate; real LLM judges are cautious and reject unsourced truths and even source-backed updates. (2) Calibrated regression tests on unrelated knowledge slow learning down without filtering out lies. (3) The original model's own prior flags claims that contradict what it knows, without a judge. It separates lies about known facts from true new facts (AUC 0.78-0.83 across three models) but not from legitimate updates of the same kind of fact (AUC 0.53). It saves 43% of judge calls, but an attacker with access to the model can choose lies that pass it. (4) If trusted sources exist that an attacker cannot plant, routing contradicted claims to evidence recovers legitimate updates and the combined system has the lowest poison rate among learned claims; when an attacker plants matching sources for 30% of the lies, the system ends up worse than the judge alone. The acceptance signals answer different questions, and none of them decides on its own whether a claim is true.