- A benchmark metric has a defined slice.
- Screening does not measure a completed breach.
- Runtime authority still needs enforcement.
The carrier already contains instructions
Unlike an ordinary retrieved page, a skill intentionally tells an agent how to work. Text-only screening therefore faces an awkward boundary: useful instructions and harmful redirections can look similar. RouteGuard studies whether internal signals during a short model probe help distinguish them before execution.
Keep the unit of evidence clear
The authors combine response-conditioned attention and hidden-state alignment from a frozen model, then evaluate on selected skill benchmarks. Their paper reports an F1 score of 0.8834 on one Skill-Inject channel slice. That is a study-specific detection score, not the chance that an arbitrary production package will be caught.
A classifier warning is evidence for review, not evidence that the flagged instruction ran or caused harm. A negative result is not an approval to grant broad permissions.
Read the authors’ limitations
The paper explicitly limits itself to pre-execution detection, controlled channel probes, representative open-weight backbones, and several paper-faithful baseline reproductions. Access to model internals may also differ between an experimental setup and the host you actually use. Check the model, threshold, false positives, corpus overlap, and action policy before transfer.
Layer the review
Inventory the whole skill, use static inspection to locate concerns, test relevant behavior with synthetic inputs, and enforce path, network, and approval boundaries when the agent acts. Document which stage each method covers. No single detector result certifies the full package, its dependencies, updates, or downstream effects.
Sources & scope
This guide combines editorial explanation and practical review suggestions. Linked sources support the underlying concepts; this site has not independently replicated their experiments.
S33