Poisoning discovery and skill selection

Selection is part of the surface

A host needs to find a relevant skill before it can use one. Names, descriptions, indexes, retrieval systems, and planner decisions influence that selection. If an unsafe package is never selected, its body may never be loaded; if selection is manipulated, it may receive a foothold in a task it should not serve.

SkillSec-Eval explicitly studies lifecycle stages including admission, retrieval, and selection. This is a useful reminder to inspect the route into execution, not only the executed code.

Review declared scope

Compare the description against the actual package. Does a narrow utility claim to be essential for unrelated tasks? Does it imply affiliation, compatibility, or privilege that its provenance does not support? Do sample tasks match the actions its scripts perform?

Keep an approved task scope with the package. Scope should help a reviewer and a selection system distinguish intended use from opportunistic activation.

Control the registry boundary

An organization can maintain a reviewed set of packages rather than letting discovery imply installation. Admission should establish source identity, ownership, permitted contexts, and update responsibility. Search ranking is not a trust score.

Separate discovery metadata from review evidence in the interface. A high-quality description and a popular listing are not proof that the package was assessed for the environment where it will run.

Test the selection step

Create routine tasks inside and outside the intended scope. Record which packages are proposed, which are loaded, and why approval was required. Include near-matching tasks and descriptions with exaggerated relevance in an isolated test registry.

An acceptable result should include correct refusal to activate unrelated packages without making the agent unusable for legitimate tasks. Keep the registry version and search configuration with the results.

Evidence to retain

• The metadata text at discovery time.
• Candidate packages and selection output.
• Admission decisions and approved scope.
• The loaded revision and runtime permissions.
• Unexpected activation patterns over time.

Selection behavior can change when the host, retrieval index, or model changes. Include those changes in the re-evaluation trigger.

Worked example: the wrong skill wins discovery

A user asks for a table of local sales totals. One skill accurately describes spreadsheet summaries; another claims to handle every business task and lists many popular keywords. The second package appears first. This may be poor metadata, malicious selection manipulation, or an unsuitable retrieval policy. The ranking alone does not establish which explanation is correct.

Work through the evidence

Freeze the candidate set and descriptions before the test. Record the actual request, retrieval results, selection rationale if available, selected revision, and files subsequently loaded. Try clearly in-scope and out-of-scope tasks. Repeat with a constrained candidate set to separate a registry problem from planner choice. Keep the host and retrieval configuration unchanged when comparing descriptions.

Decision criteria and a useful review record

Require consistency between advertised purpose, implementation, and approved scope. Promotion into the reviewed library should be separate from becoming discoverable. A package selected for an unrelated task should not gain extra permissions as a consequence. Retain negative examples: they show where the package should not activate, which a list of successful demonstrations alone cannot establish.