Sample content: illustrative entries and numbers, not research results
I measure how language-model defenses hold up once the attacker adapts.
A prompt-injection detector calibrated to a fixed false-positive budget catches nearly every static attack, and almost none from an attacker who optimizes against its score.
Adaptive attacks detected at the calibrated threshold
Re-run under one shared query budget, the defense with the best published robustness number ranks last of those compared.
Attack success on the best-reported defense at the shared budget
Its static-evaluation numbers reproduce within a tolerance declared in advance; its claim of robustness to adaptive attacks does not.
Claims that hold within the declared tolerance