A defence that refuses to answer anything is perfectly safe and useless. So in the RAG poisoning study I judged every defence on two numbers: how much of the attack it stops, and how much accuracy it costs on clean questions, where nothing is wrong.
The setup: 10,000 Wikipedia paragraphs from SQuAD, one planted passage at a time, three open models running locally at temperature 0 (qwen3 8B, Llama 3.1 8B, Gemma 3 12B), top 5 passages retrieved, and 783 clean questions (383 for Gemma) to price each defence on.
The instruction filter: free, and narrow
This drops instruction-like sentences from retrieved text before the model sees it. It stopped every injection, for all three models, including Gemma, which had followed the planted instruction 56% of the time. It cost 0.0 points on clean questions.
It did nothing for false facts. A false fact doesn't contain an instruction, so there's nothing to filter. Uptake stayed at 42%, 21% and 42%.
Quote your source: the prediction that failed
Asking the model to quote the passage it answers from sounds like it should make it more careful. Before the run I predicted it would reduce uptake a little at a small clean cost.
It did cut injections, to 2 to 4%. On false facts it was mixed. qwen3 went from 42% to 29%. Llama barely moved, from 21% to 19%, and lost 8.9 points of clean accuracy. Gemma went the wrong way: it repeated the false fact more, 56% instead of 42%.
So the same prompt change helped one model, cost another, and made the third worse. I would not have guessed that, and I didn't.
Source agreement: safe, and unusable as built
This keeps an answer only if at least two retrieved passages contain it, and abstains otherwise. It cut single false facts to 2 to 4%. That part worked.
It cost 55.2 to 59.5 points of clean accuracy, because it abstained on 80 to 84% of clean questions. In this corpus most answers come from one paragraph, so most true answers have only one source. And it failed against five copies of the false fact, which agree with each other by design: uptake stayed at 100%, 85% and 94%.
I predicted a large cost here, over 20 points. It was much larger. This rule is a crude string check. A model-based agreement check might do far better on accuracy. As built, it shows the trade-off, not a fix.
Other predictions that were wrong
- I expected one false passage to be adopted 30 to 70% of the time. Llama came in under that, at 21%.
- I expected injections to be obeyed less often than false facts. For Gemma it was the reverse: 56% against 42%.
A cross-encoder reranker was planned as a fourth defence but not run. No reranker model was available locally.
What I'd do in practice
- Strip instructions from retrieved text. It was free here and stopped every injection.
- Price every defence on clean traffic before turning it on. The clean cost is what you pay every day, attack or not.
- Test a prompt change per model. The same change helped one model and hurt another.
- Watch for duplicated claims, not just false ones. Five agreeing copies beat the models and beat the agreement rule.
Limits
Small local models (8B and 12B, 4-bit), temperature 0, one corpus, one retriever, templated poisons, 48 targets, so the attack rates carry wide intervals. Gemma ran a reduced design. None of this is about hosted frontier models.
Every defence table is in the case study, and the code is on GitHub.