What marketplace studies can actually tell us

Three collections answer different questions

A marketplace scan asks what its detection process found in a collected sample. A behaviorally verified malicious corpus asks which cases met a stronger confirmation procedure. An adversarial benchmark deliberately assembles cases for testing. Their denominators are not interchangeable.

Keep findings attached to their collection

Compare the collection period, registries, package versions, deduplication, labels, and verification method. Repeated publishing under different names can affect counts. A study’s historical findings do not establish which packages remain available today.

Use complementary evidence

The registry measurement study and Snyk investigation offer different collection and analysis contexts. Read both as source-specific evidence. A result labeled vulnerable can include accidental weaknesses; malicious intent requires further support. Do not combine their counts into a market total.

Turn reading into a bounded decision

For a package you plan to use, obtain its exact revision and inspect its own behavior. Ecosystem statistics can justify a review process, but cannot decide the status of an individual artifact. Retain observed evidence, runtime authority, and unresolved questions in the review record.

Worked example: three reports are not three independent populations

A campaign report counts published artifacts, a later vendor audit counts author-and-skill identifiers, and a benchmark assembles selected examples from several reports. Their totals can overlap. Adding them would not produce a count of unique malicious packages, unique operators, or affected people.

Work through the evidence

Write down the unit, collection date, source registry, deduplication rule, label method, and outcome level. Ask whether “malicious” means a static pattern, manually confirmed behavior, or observed effect in a particular environment. Record removals as dated statements from the source rather than assuming those statements describe the current registry.

Decision criteria and a useful review record

Use ecosystem studies to inform review priorities without deciding the status of an individual package by prevalence. A large sample can still miss attacks; a curated sample can be deliberately unrepresentative. In the case archive, package findings, controlled experiments, and measurement studies are explicitly separated so a reader can compare the evidence without collapsing those categories.