用 Jev 跑提示注入越狱基准,对比 Meta 等护栏分类器
作者把 Jev 当越狱/提示注入分类器实测:在新攻击集上胜过现成护栏,但旧集和低误报预算下会输,内部隐蔽攻击也失败。

Sharing some prompt injection "jailbreak" benchmarks I've been running with Jev, competing against a bunch of standard guardrail classifiers including Meta's. Part of it is a standard open suite that's getting dated. Jev wins that one outright, and wins the newest attack set too. It loses on the older sets and can't run at a tight false-alarm budget. It failed some of our internal benchmarks when something more nefarious is going on & is obscured, but there's really not any good off the shelf classifier that could be used for those type of scenarios. You can see a glimpse of this with how every model performed poorly at allenai's WildJailbreak. Since the poker eval I've been trying to pin down where Jev fits. My hunch is perfect-information calls with little strategy in them. Still testing it.